Mathwords logoReference LibraryMathwords

Significance Level (Alpha)

Significance level (alpha, written α\alpha) is the cutoff you choose before running a hypothesis test — if your p-value falls below α\alpha, you reject the null hypothesis. The most common significance level is 0.05, meaning you're willing to accept a 5% chance of rejecting a true null hypothesis.

The significance level α\alpha is the maximum probability of committing a Type I error — rejecting the null hypothesis when it is actually true. It is set before data collection and defines the rejection region of the test. A result is called statistically significant when the observed p-value satisfies p≤αp \leq \alpha. Common choices for α\alpha include 0.01, 0.05, and 0.10, depending on the context and consequences of a false rejection.

Key Formula

Reject H0 if p-value≤α\text{Reject } H_0 \text{ if } p\text{-value} \leq \alpha
Where:
  • H0H₀ = the null hypothesis
  • p−valuep-value = the probability of observing results at least as extreme as the data, assuming the null hypothesis is true
  • αα = the significance level (threshold for rejection)

Worked Example

Problem: A company claims that 50% of customers prefer their new product. You survey 200 customers and find that 58% prefer it. A one-proportion z-test gives a p-value of 0.024. Using a significance level of α=0.05\alpha = 0.05, should you reject the null hypothesis?
Step 1: State the significance level chosen before the test.
α=0.05\alpha = 0.05
Step 2: Identify the p-value from the test.
p-value=0.024p\text{-value} = 0.024
Step 3: Compare the p-value to α\alpha.
0.024≤0.050.024 \leq 0.05
Step 4: Since the p-value is less than or equal to α\alpha, reject H0H_0. There is statistically significant evidence that the true proportion of customers who prefer the new product differs from 50%.
Answer: At the α=0.05\alpha = 0.05 significance level, we reject the null hypothesis because the p-value of 0.024 is below the threshold.

Why It Matters

Every hypothesis test in statistics requires a significance level, so understanding α\alpha is essential for interpreting research results. In medicine, a stricter α\alpha like 0.01 might be used because a false conclusion could harm patients, while in exploratory social science research, 0.10 might be acceptable. Choosing α\alpha is a deliberate decision about how much risk of a wrong rejection you're willing to tolerate.

Common Mistakes

Mistake: Choosing or changing α\alpha after seeing the p-value to get the result you want.
Correction: The significance level must be set before conducting the test. Adjusting it afterward undermines the integrity of the test and is considered bad statistical practice.
Mistake: Interpreting α=0.05\alpha = 0.05 as a 5% chance that the null hypothesis is true.
Correction: Alpha is the probability of rejecting a true null hypothesis (Type I error rate), not the probability that the null hypothesis itself is true or false. These are different concepts.

Related Terms

  • Null Hypothesis — The hypothesis you reject or fail to reject using alpha
  • p-value — Compared directly against alpha to make a decision