Mathwords logoMathwords

Gram-Schmidt Orthonormalization — Definition, Formula & Examples

Gram-Schmidt orthonormalization is a procedure that takes a set of linearly independent vectors and produces a set of orthonormal vectors (mutually perpendicular and each of unit length) that span the same subspace.

Given a set of linearly independent vectors {v1,v2,…,vk}\{\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_k\} in an inner product space, the Gram-Schmidt process constructs an orthonormal set {e1,e2,…,ek}\{\mathbf{e}_1, \mathbf{e}_2, \ldots, \mathbf{e}_k\} such that for each jj, span⁡(e1,…,ej)=span⁡(v1,…,vj)\operatorname{span}(\mathbf{e}_1, \ldots, \mathbf{e}_j) = \operatorname{span}(\mathbf{v}_1, \ldots, \mathbf{v}_j).

Key Formula

uk=vk−∑j=1k−1(vk⋅ej) ej,ek=uk∥uk∥\mathbf{u}_k = \mathbf{v}_k - \sum_{j=1}^{k-1} (\mathbf{v}_k \cdot \mathbf{e}_j)\,\mathbf{e}_j, \qquad \mathbf{e}_k = \frac{\mathbf{u}_k}{\|\mathbf{u}_k\|}
Where:
  • vk\mathbf{v}_k = The k-th original linearly independent vector
  • ej\mathbf{e}_j = The j-th orthonormal vector already computed
  • uk\mathbf{u}_k = Intermediate orthogonal (but not yet normalized) vector
  • ∥uk∥\|\mathbf{u}_k\| = The Euclidean norm (length) of the intermediate vector

How It Works

At each step you subtract from the current vector its projections onto all previously computed orthonormal vectors, then normalize the result. The first orthonormal vector is simply v1\mathbf{v}_1 divided by its norm. For each subsequent vector vk\mathbf{v}_k, you remove the components along e1,…,ek−1\mathbf{e}_1, \ldots, \mathbf{e}_{k-1} to get an orthogonal vector uk\mathbf{u}_k, then divide by ∥uk∥\|\mathbf{u}_k\| to obtain ek\mathbf{e}_k. This guarantees every pair of output vectors has a dot product of zero and each has length 1.

Worked Example

Problem: Apply Gram-Schmidt orthonormalization to the vectors v1=(1,1,0)\mathbf{v}_1 = (1, 1, 0) and v2=(1,0,1)\mathbf{v}_2 = (1, 0, 1).
Step 1: Normalize the first vector.
e1=v1∥v1∥=(1,1,0)2=(12, 12, 0)\mathbf{e}_1 = \frac{\mathbf{v}_1}{\|\mathbf{v}_1\|} = \frac{(1,1,0)}{\sqrt{2}} = \left(\tfrac{1}{\sqrt{2}},\,\tfrac{1}{\sqrt{2}},\,0\right)
Step 2: Subtract the projection of v2\mathbf{v}_2 onto e1\mathbf{e}_1.
v2⋅e1=12,u2=(1,0,1)−12(12, 12, 0)=(12, −12, 1)\mathbf{v}_2 \cdot \mathbf{e}_1 = \frac{1}{\sqrt{2}}, \qquad \mathbf{u}_2 = (1,0,1) - \frac{1}{\sqrt{2}}\left(\tfrac{1}{\sqrt{2}},\,\tfrac{1}{\sqrt{2}},\,0\right) = \left(\tfrac{1}{2},\,-\tfrac{1}{2},\,1\right)
Step 3: Normalize the result.
∥u2∥=14+14+1=32,e2=u23/2=(16, −16, 26)\|\mathbf{u}_2\| = \sqrt{\tfrac{1}{4}+\tfrac{1}{4}+1} = \sqrt{\tfrac{3}{2}}, \qquad \mathbf{e}_2 = \frac{\mathbf{u}_2}{\sqrt{3/2}} = \left(\tfrac{1}{\sqrt{6}},\,-\tfrac{1}{\sqrt{6}},\,\tfrac{2}{\sqrt{6}}\right)
Answer: The orthonormal set is e1=(12, 12, 0)\mathbf{e}_1 = \left(\tfrac{1}{\sqrt{2}},\,\tfrac{1}{\sqrt{2}},\,0\right) and e2=(16, −16, 26)\mathbf{e}_2 = \left(\tfrac{1}{\sqrt{6}},\,-\tfrac{1}{\sqrt{6}},\,\tfrac{2}{\sqrt{6}}\right).

Why It Matters

Gram-Schmidt is the foundation of QR factorization, one of the most widely used matrix decompositions in numerical computing. It appears throughout signal processing, computer graphics, and statistics whenever you need an orthonormal basis — for example, in least-squares regression or principal component analysis.

Common Mistakes

Mistake: Forgetting to normalize after each subtraction step, producing orthogonal but not orthonormal vectors.
Correction: Always divide uk\mathbf{u}_k by its norm immediately before using it in subsequent projection calculations. The projection formula shown above assumes each ej\mathbf{e}_j already has unit length.