Projection Matrix — Definition, Formula & Examples

A projection matrix is a square matrix

P

that, when multiplied by a vector, gives the orthogonal projection of that vector onto a subspace. Applying

P

twice has the same effect as applying it once, meaning

P^2 = P

A matrix $P$ is a projection matrix if $P^2 = P$ (idempotent). It is an orthogonal projection matrix if, additionally, $P = P^T$ (symmetric). Given a matrix $A$ whose columns span the target subspace, the orthogonal projection matrix onto the column space of $A$ is $P = A(A^T A)^{-1}A^T$ .

Key Formula

P = A(A^T A)^{-1}A^T

Where:

$P$ = The orthogonal projection matrix onto the column space of A
$A$ = Matrix whose columns form a basis for the target subspace
$A^T$ = Transpose of A
$(A^T A)^{-1}$ = Inverse of the Gram matrix, which exists when A has linearly independent columns

How It Works

To project a vector

\mathbf{b}

onto the column space of a matrix

A

, you compute

P\mathbf{b}

where

P = A(A^T A)^{-1}A^T

. The result

\hat{\mathbf{b}} = P\mathbf{b}

is the closest point in the column space to

\mathbf{b}

. The residual

\mathbf{b} - \hat{\mathbf{b}}

is orthogonal to every column of

A

. This is the geometric heart of least squares regression: when

A\mathbf{x} = \mathbf{b}

has no exact solution, the projection gives the best approximation.

Worked Example

Problem: Find the projection matrix that projects vectors in R² onto the line spanned by a = [1, 2]ᵀ.

Compute AᵀA: Here A is just the column vector a. Compute the dot product.

A^T A = \begin{bmatrix}1 & 2\end{bmatrix}\begin{bmatrix}1\\2\end{bmatrix} = 5

Compute (AᵀA)⁻¹: Since AᵀA is a scalar, its inverse is simply 1/5.

(A^T A)^{-1} = \frac{1}{5}

Compute P = A(AᵀA)⁻¹Aᵀ: Multiply the column vector by 1/5 by the row vector.

P = \frac{1}{5}\begin{bmatrix}1\\2\end{bmatrix}\begin{bmatrix}1 & 2\end{bmatrix} = \frac{1}{5}\begin{bmatrix}1 & 2\\2 & 4\end{bmatrix}

Answer: The projection matrix is

P = \frac{1}{5}\begin{bmatrix}1 & 2\\2 & 4\end{bmatrix}

. You can verify:

P^2 = P

and

P^T = P

Why It Matters

Projection matrices are the foundation of least squares solutions. In statistics and data science, every ordinary least squares regression computes a projection of observed data onto the column space of the design matrix. Understanding projections also clarifies concepts like residuals and the hat matrix in regression diagnostics.

Common Mistakes

Mistake: Forgetting to check that the columns of A are linearly independent before computing (AᵀA)⁻¹.

Correction: The formula P = A(AᵀA)⁻¹Aᵀ requires AᵀA to be invertible, which happens only when A has linearly independent columns. If columns are dependent, remove redundant columns first or use the pseudoinverse.