General Linear Model — Definition, Formula & Example
The general linear model is one framework that covers linear regression, t-tests, and ANOVA. It predicts a numeric outcome from a weighted sum of predictors plus random error, and each predictor can be numeric or a coded group label.
In its basic form, the general linear model states that an response vector satisfies . Here is a known design matrix and is a vector of unknown parameters. The errors are typically assumed independent with . The model is linear in the parameters , not necessarily in the original variables, so a term like can be a column of . When has full column rank, the least-squares estimate is .
Key Formula
Where:
- = Vector of the n observed outcomes
- = Design matrix: a column of 1s for the intercept, then one column per predictor (numeric values or 0/1 group codes)
- = Vector of unknown coefficients, one per column of X
- = Vector of random errors, usually assumed independent and normal with mean 0 and constant variance
- = Least-squares estimate of the coefficients
Worked Example
Problem: Six students record their one-way commute times in minutes. The three who bike take 20, 22, and 24 minutes. The three who ride the bus take 30, 32, and 34 minutes. Code bus = 1 and bike = 0, then fit the general linear model by least squares. What do the coefficients mean?
Step 1: Turn the group label into a 0/1 predictor . Then find both means. Three of the six values of are 1, and the commute times sum to 162.
Step 2: Compute the least-squares slope. Each bike student contributes to the numerator, and each bus student contributes . Every value of equals 0.25.
Step 3: Find the intercept from the fact that the fitted line passes through the point of means.
Answer: The fitted model is . The intercept, 22 minutes, is the mean bike time. The slope, 10 minutes, is exactly the bus mean (32) minus the bike mean (22). A regression on a 0/1 variable is the same comparison that a pooled (equal-variance) two-sample t-test makes, which is why both methods are special cases of the general linear model.
Why It Matters
In a college statistics or research-methods course, the general linear model shows that t-tests, ANOVA, ANCOVA, and multiple regression are one method. Software such as R's lm() function fits all of them the same way. Psychology, biology, and economics researchers use it to adjust a group comparison for numeric covariates like age or income.
Common Mistakes
Mistake: Confusing the general linear model with the generalized linear model (GLM).
Correction: The general linear model assumes a numeric outcome with normal, constant-variance errors. The generalized linear model extends it with a link function and other error distributions, as in logistic regression for yes/no outcomes. The general linear model is the special case with an identity link and normal errors.
