Mathwords logoMathwords

Statistically Significant — Definition, Formula & Examples

Statistically significant describes a sample result that would be unlikely to happen by random chance alone if the null hypothesis were true. When a result is significant, you have convincing evidence against the null hypothesis.

A result is statistically significant at level α\alpha if its p-value satisfies p-value≤α\text{p-value} \le \alpha. The p-value is computed assuming the null hypothesis H0H_0 is true. Equivalently, the test statistic falls in the rejection region bounded by the critical value(s). The significance level α\alpha must be chosen before the data are examined. It equals the probability of a Type I error, which means rejecting H0H_0 when H0H_0 is actually true.

Key Formula

p-value≤α  ⟹  statistically significant (reject H0)\text{p-value} \le \alpha \;\Longrightarrow\; \text{statistically significant (reject } H_0\text{)}
Where:
  • p-value\text{p-value} = Probability, assuming H0H_0 is true, of getting a test statistic at least as extreme as the one observed
  • α\alpha = Significance level chosen before the study, such as 0.05 or 0.01
  • H0H_0 = The null hypothesis, a claim of no effect or no difference

How It Works

Before collecting data, you choose a significance level α\alpha. AP Statistics most often uses α=0.05\alpha = 0.05. You then compute a test statistic from your sample and find its p-value. If the p-value is less than or equal to α\alpha, the result is statistically significant, and you reject H0H_0 in favor of the alternative hypothesis HaH_a. If the p-value is greater than α\alpha, you fail to reject H0H_0. That outcome means the evidence is not strong enough. It does not prove H0H_0 is true.

Worked Example

Problem: A coin is suspected of landing heads more than half the time. You flip it 100 times and get 60 heads. Test H0:p=0.5H_0: p = 0.5 against Ha:p>0.5H_a: p > 0.5 at α=0.05\alpha = 0.05. Is the result statistically significant?
Step 1: Find the standard error of the sample proportion. Use p0=0.5p_0 = 0.5 from the null hypothesis.
SE=p0(1−p0)n=0.5(0.5)100=0.05SE = \sqrt{\frac{p_0(1-p_0)}{n}} = \sqrt{\frac{0.5(0.5)}{100}} = 0.05
Step 2: Compute the z test statistic. The sample proportion is p^=60/100=0.60\hat{p} = 60/100 = 0.60.
z=p^−p0SE=0.60−0.500.05=2z = \frac{\hat{p} - p_0}{SE} = \frac{0.60 - 0.50}{0.05} = 2
Step 3: Find the one-sided p-value. It is the area under the standard normal curve to the right of z=2z = 2.
p-value=P(Z≥2)≈0.0228\text{p-value} = P(Z \ge 2) \approx 0.0228
Step 4: Compare the p-value to α\alpha.
0.0228≤0.050.0228 \le 0.05
Answer: Yes. The p-value of about 0.0228 is less than α=0.05\alpha = 0.05, so the result is statistically significant. You reject H0H_0. There is convincing evidence that the coin lands heads more than half the time.

Why It Matters

On the AP Statistics exam, every significance test ends with a conclusion that compares the p-value to α and is stated in context. You lose credit if either part is missing. Outside the classroom, medical researchers use statistical significance to decide whether a new drug beats a placebo, and tech companies use it in A/B tests to decide whether a website change actually raised sales.

Common Mistakes

Mistake: Thinking 'statistically significant' means the effect is large or important.
Correction: Significance only says the result is unlikely to be due to chance alone. With a very large sample, a tiny and practically meaningless difference can still be significant. Check the actual size of the effect, for example with a confidence interval.
Mistake: Concluding that a non-significant result proves the null hypothesis is true.
Correction: A p-value above α means you fail to reject H0H_0. The data simply do not give strong enough evidence against it. Write 'there is not convincing evidence that...' rather than 'this proves that...'.

Related Terms