Mathwords logoMathwords

General Linear Model — Definition, Formula & Example

The general linear model is one framework that covers linear regression, t-tests, and ANOVA. It predicts a numeric outcome from a weighted sum of predictors plus random error, and each predictor can be numeric or a coded group label.

In its basic form, the general linear model states that an n×1n \times 1 response vector satisfies y=Xβ+ε\mathbf{y} = X\boldsymbol{\beta} + \boldsymbol{\varepsilon}. Here XX is a known n×(p+1)n \times (p+1) design matrix and β\boldsymbol{\beta} is a vector of unknown parameters. The errors are typically assumed independent with ε∼N(0,σ2I)\boldsymbol{\varepsilon} \sim N(\mathbf{0}, \sigma^2 I). The model is linear in the parameters βj\beta_j, not necessarily in the original variables, so a term like x2x^2 can be a column of XX. When XX has full column rank, the least-squares estimate is β^=(X⊤X)−1X⊤y\hat{\boldsymbol{\beta}} = (X^{\top}X)^{-1}X^{\top}\mathbf{y}.

Key Formula

y=Xβ+ε,β^=(X⊤X)−1X⊤y\mathbf{y} = X\boldsymbol{\beta} + \boldsymbol{\varepsilon}, \qquad \hat{\boldsymbol{\beta}} = (X^{\top}X)^{-1}X^{\top}\mathbf{y}
Where:
  • y\mathbf{y} = Vector of the n observed outcomes
  • XX = Design matrix: a column of 1s for the intercept, then one column per predictor (numeric values or 0/1 group codes)
  • β\boldsymbol{\beta} = Vector of unknown coefficients, one per column of X
  • ε\boldsymbol{\varepsilon} = Vector of random errors, usually assumed independent and normal with mean 0 and constant variance
  • β^\hat{\boldsymbol{\beta}} = Least-squares estimate of the coefficients

Worked Example

Problem: Six students record their one-way commute times in minutes. The three who bike take 20, 22, and 24 minutes. The three who ride the bus take 30, 32, and 34 minutes. Code bus = 1 and bike = 0, then fit the general linear model y=β0+β1x+εy = \beta_0 + \beta_1 x + \varepsilon by least squares. What do the coefficients mean?
Step 1: Turn the group label into a 0/1 predictor xx. Then find both means. Three of the six values of xx are 1, and the commute times sum to 162.
xˉ=36=0.5,yˉ=1626=27\bar{x} = \frac{3}{6} = 0.5, \qquad \bar{y} = \frac{162}{6} = 27
Step 2: Compute the least-squares slope. Each bike student contributes (−0.5)(y−27)(-0.5)(y-27) to the numerator, and each bus student contributes (0.5)(y−27)(0.5)(y-27). Every value of (x−xˉ)2(x - \bar{x})^2 equals 0.25.
β^1=∑(x−xˉ)(y−yˉ)∑(x−xˉ)2=7.5+7.56(0.25)=151.5=10\hat{\beta}_1 = \frac{\sum (x-\bar{x})(y-\bar{y})}{\sum (x-\bar{x})^2} = \frac{7.5 + 7.5}{6(0.25)} = \frac{15}{1.5} = 10
Step 3: Find the intercept from the fact that the fitted line passes through the point of means.
β^0=yˉ−β^1xˉ=27−10(0.5)=22\hat{\beta}_0 = \bar{y} - \hat{\beta}_1\bar{x} = 27 - 10(0.5) = 22
Answer: The fitted model is y^=22+10x\hat{y} = 22 + 10x. The intercept, 22 minutes, is the mean bike time. The slope, 10 minutes, is exactly the bus mean (32) minus the bike mean (22). A regression on a 0/1 variable is the same comparison that a pooled (equal-variance) two-sample t-test makes, which is why both methods are special cases of the general linear model.

Why It Matters

In a college statistics or research-methods course, the general linear model shows that t-tests, ANOVA, ANCOVA, and multiple regression are one method. Software such as R's lm() function fits all of them the same way. Psychology, biology, and economics researchers use it to adjust a group comparison for numeric covariates like age or income.

Common Mistakes

Mistake: Confusing the general linear model with the generalized linear model (GLM).
Correction: The general linear model assumes a numeric outcome with normal, constant-variance errors. The generalized linear model extends it with a link function and other error distributions, as in logistic regression for yes/no outcomes. The general linear model is the special case with an identity link and normal errors.

Related Terms