Mathwords logoMathwords

Gram Matrix — Definition, Formula & Examples

A Gram matrix is a matrix of all pairwise inner products of a set of vectors. If your vectors are the columns of a matrix AA, the Gram matrix is G=A⊤AG = A^\top A.

Given vectors v1,v2,…,vn\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_n in an inner product space, the Gram matrix GG is the n×nn \times n matrix whose (i,j)(i,j)-entry is Gij=⟨vi,vj⟩G_{ij} = \langle \mathbf{v}_i, \mathbf{v}_j \rangle. When these vectors form the columns of a real matrix A∈Rm×nA \in \mathbb{R}^{m \times n}, the Gram matrix equals G=A⊤AG = A^\top A.

Key Formula

G=A⊤Awhere Gij=vi⋅vjG = A^\top A \quad \text{where } G_{ij} = \mathbf{v}_i \cdot \mathbf{v}_j
Where:
  • AA = An m × n real matrix whose columns are the vectors v₁, …, vₙ
  • GG = The n × n Gram matrix of inner products
  • GijG_{ij} = The dot product of the i-th and j-th column vectors of A

How It Works

Each entry GijG_{ij} tells you the dot product of the ii-th and jj-th vectors. Diagonal entries Gii=∥vi∥2G_{ii} = \|\mathbf{v}_i\|^2 give the squared lengths. Off-diagonal entries measure how aligned two vectors are. The Gram matrix is always symmetric and positive semi-definite. Its determinant equals zero exactly when the vectors are linearly dependent, which makes it a direct test for linear independence.

Worked Example

Problem: Find the Gram matrix for the columns of A = [[1, 2], [3, 0], [0, 1]].
Step 1: Write out A and compute Aᵀ.
A=(123001),A⊤=(130201)A = \begin{pmatrix} 1 & 2 \\ 3 & 0 \\ 0 & 1 \end{pmatrix}, \quad A^\top = \begin{pmatrix} 1 & 3 & 0 \\ 2 & 0 & 1 \end{pmatrix}
Step 2: Multiply Aᵀ A to get the 2×2 Gram matrix.
G=A⊤A=(1⋅1+3⋅3+0⋅01⋅2+3⋅0+0⋅12⋅1+0⋅3+1⋅02⋅2+0⋅0+1⋅1)=(10225)G = A^\top A = \begin{pmatrix} 1\cdot1+3\cdot3+0\cdot0 & 1\cdot2+3\cdot0+0\cdot1 \\ 2\cdot1+0\cdot3+1\cdot0 & 2\cdot2+0\cdot0+1\cdot1 \end{pmatrix} = \begin{pmatrix} 10 & 2 \\ 2 & 5 \end{pmatrix}
Step 3: Interpret the result. The diagonal entries 10 and 5 are the squared norms of the two column vectors. The off-diagonal entry 2 is their dot product.
Answer: The Gram matrix is G=(10225)G = \begin{pmatrix} 10 & 2 \\ 2 & 5 \end{pmatrix}.

Why It Matters

The Gram matrix appears in least-squares problems, where the normal equations A⊤Ax=A⊤bA^\top A \mathbf{x} = A^\top \mathbf{b} use it directly. It is also central to kernel methods in machine learning, where you compute inner products without explicitly working in high-dimensional feature spaces.

Common Mistakes

Mistake: Computing AA⊤A A^\top instead of A⊤AA^\top A and calling it the Gram matrix of the columns.
Correction: AA⊤A A^\top gives inner products of the row vectors (or equivalently columns of A⊤A^\top). For the Gram matrix of the columns of AA, you need A⊤AA^\top A.