Mathwords logoMathwords

Multicollinearity — Definition & Example

Multicollinearity is what happens when two or more explanatory variables in a multiple regression carry nearly the same information. The model can still predict well, but it has trouble telling how much of the effect belongs to each variable.

In a multiple regression model y^=b0+b1x1+⋯+bkxk\hat{y} = b_0 + b_1x_1 + \cdots + b_kx_k, multicollinearity is present when some explanatory variable xjx_j is approximately a linear combination of the other explanatory variables (plus a constant). If the relationship is exact (perfect multicollinearity), the least-squares coefficients are not unique. If it is approximate, the coefficients exist but have inflated standard errors.

How It Works

Look for warning signs: a large correlation between two predictors, coefficients whose signs contradict common sense, or a model with a high R2R^2 whose individual slopes are not statistically significant. Software often reports a variance inflation factor (VIF) for each predictor. Values above about 5 or 10 are common red flags. To fix the problem, drop one of the redundant predictors, combine them into a single measure, or collect data in which the predictors vary more independently.

Worked Example

Problem: A lumber store models the price of a board (in dollars) using two predictors: x1x_1 = length in inches and x2x_2 = length in feet. One fitted model is y^=2+0.5x1\hat{y} = 2 + 0.5x_1. Show that very different coefficient choices give the same predicted price for a 48-inch (4-foot) board.
Step 1: Use only inches. Substitute x1=48x_1 = 48.
y^=2+0.5(48)=26\hat{y} = 2 + 0.5(48) = 26
Step 2: Split the effect between inches and feet. Since x2=x1/12x_2 = x_1/12, the term 3x23x_2 equals 0.25x10.25x_1. Substitute x1=48x_1 = 48 and x2=4x_2 = 4.
y^=2+0.25(48)+3(4)=2+12+12=26\hat{y} = 2 + 0.25(48) + 3(4) = 2 + 12 + 12 = 26
Step 3: Try a model with a negative coefficient on inches. Here 12x2=x112x_2 = x_1, so −0.5x1+12x2=0.5x1-0.5x_1 + 12x_2 = 0.5x_1.
y^=2−0.5(48)+12(4)=2−24+48=26\hat{y} = 2 - 0.5(48) + 12(4) = 2 - 24 + 48 = 26
Answer: All three models predict $26. Because feet and inches are perfectly collinear, the data cannot decide among the coefficients. The coefficient on inches could be 0.5, 0.25, or even negative.

Why It Matters

In economics and medical research, analysts often want to say how much one factor matters while holding the others fixed. Height and arm span are one example, and income and education are another. When those factors are highly correlated, the individual coefficients cannot answer that question reliably, even if the model's predictions are accurate.

Common Mistakes

Mistake: Concluding that a predictor has no effect because its slope is not significant in a model with a strongly correlated partner variable.
Correction: Multicollinearity inflates standard errors and widens confidence intervals, so real effects can look insignificant. Refit the model with one of the correlated predictors removed, or test the predictors together, before you conclude anything.

Related Terms