Power of a Test — Definition, Formula & Examples
Power of a test is the probability that a hypothesis test correctly rejects the null hypothesis when it is actually false. In other words, it measures how good your test is at detecting a real effect.
The power of a statistical test is defined as , where is the probability of a Type II error (failing to reject a false null hypothesis). Power depends on the significance level , the sample size , and the true value of the parameter being tested.
Key Formula
Where:
- = Probability of a Type II error (failing to reject a false null hypothesis)
- = Probability of correctly rejecting a false null hypothesis
How It Works
A test with high power is unlikely to miss a real effect. Power ranges from 0 to 1, and researchers typically aim for power of at least 0.80. You can increase power by increasing sample size, increasing the significance level , or when the true parameter value is farther from the null hypothesis value. In AP Statistics, you won't usually compute power directly, but you need to understand what affects it and how to interpret it.
Worked Example
Problem: A drug company tests whether a new medication lowers blood pressure. They use a significance level of 0.05. If the true mean reduction is 10 mmHg, the probability of failing to reject the null hypothesis (Type II error) is 0.15. What is the power of the test?
Identify β: The probability of a Type II error is given as 0.15.
Calculate power: Subtract β from 1.
Answer: The power of the test is 0.85, meaning there is an 85% chance of correctly detecting the 10 mmHg reduction if it truly exists.
Why It Matters
On the AP Statistics exam, free-response questions frequently ask how changing sample size or significance level affects power. In research and medicine, underpowered studies waste resources and can miss treatments that actually work, so scientists use power analysis to plan appropriate sample sizes before collecting data.
Common Mistakes
Mistake: Confusing power with the significance level α
Correction: The significance level α is the probability of a Type I error (rejecting a true null hypothesis). Power is about correctly rejecting a false null hypothesis. They are related — increasing α does increase power — but they measure different things.