Multicollinearity — Definition & Example
Multicollinearity is what happens when two or more explanatory variables in a multiple regression carry nearly the same information. The model can still predict well, but it has trouble telling how much of the effect belongs to each variable.
In a multiple regression model , multicollinearity is present when some explanatory variable is approximately a linear combination of the other explanatory variables (plus a constant). If the relationship is exact (perfect multicollinearity), the least-squares coefficients are not unique. If it is approximate, the coefficients exist but have inflated standard errors.
How It Works
Look for warning signs: a large correlation between two predictors, coefficients whose signs contradict common sense, or a model with a high whose individual slopes are not statistically significant. Software often reports a variance inflation factor (VIF) for each predictor. Values above about 5 or 10 are common red flags. To fix the problem, drop one of the redundant predictors, combine them into a single measure, or collect data in which the predictors vary more independently.
Worked Example
Problem: A lumber store models the price of a board (in dollars) using two predictors: = length in inches and = length in feet. One fitted model is . Show that very different coefficient choices give the same predicted price for a 48-inch (4-foot) board.
Step 1: Use only inches. Substitute .
Step 2: Split the effect between inches and feet. Since , the term equals . Substitute and .
Step 3: Try a model with a negative coefficient on inches. Here , so .
Answer: All three models predict $26. Because feet and inches are perfectly collinear, the data cannot decide among the coefficients. The coefficient on inches could be 0.5, 0.25, or even negative.
Why It Matters
In economics and medical research, analysts often want to say how much one factor matters while holding the others fixed. Height and arm span are one example, and income and education are another. When those factors are highly correlated, the individual coefficients cannot answer that question reliably, even if the model's predictions are accurate.
Common Mistakes
Mistake: Concluding that a predictor has no effect because its slope is not significant in a model with a strongly correlated partner variable.
Correction: Multicollinearity inflates standard errors and widens confidence intervals, so real effects can look insignificant. Refit the model with one of the correlated predictors removed, or test the predictors together, before you conclude anything.
