Singular Value — Definition, Formula & Examples

A singular value of a matrix is a non-negative number that describes how much the matrix stretches input vectors along a particular direction. Every

m \times n

matrix has singular values, obtained through singular value decomposition (SVD).

The singular values of an $m \times n$ matrix $A$ are the non-negative square roots of the eigenvalues of $A^T A$ (or equivalently $A A^T$ ). They are conventionally ordered as $\sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_p \geq 0$ , where $p = \min(m, n)$ . In the SVD $A = U \Sigma V^T$ , these values appear on the diagonal of $\Sigma$ .

Key Formula

\sigma_i = \sqrt{\lambda_i(A^T A)}

Where:

$\sigma_i$ = The $i$-th singular value of matrix $A$
$\lambda_i(A^T A)$ = The $i$-th eigenvalue of $A^T A$
$A$ = An $m \times n$ real matrix

How It Works

To find the singular values of a matrix

A

, you compute

A^T A

, find its eigenvalues, and take their non-negative square roots. The number of nonzero singular values equals the rank of

A

. Larger singular values correspond to directions along which

A

has the greatest stretching effect, while singular values near zero indicate directions that are nearly collapsed. This makes singular values central to understanding the "geometry" of a linear transformation.

Worked Example

Problem: Find the singular values of A = [[3, 0], [0, 2]].

Step 1: Compute the product

A^T A

A^T A = \begin{bmatrix} 3 & 0 \\ 0 & 2 \end{bmatrix}\begin{bmatrix} 3 & 0 \\ 0 & 2 \end{bmatrix} = \begin{bmatrix} 9 & 0 \\ 0 & 4 \end{bmatrix}

Step 2: Find the eigenvalues of

A^T A

. Since the matrix is diagonal, the eigenvalues are the diagonal entries.

\lambda_1 = 9, \quad \lambda_2 = 4

Step 3: Take the square roots to get the singular values.

\sigma_1 = \sqrt{9} = 3, \quad \sigma_2 = \sqrt{4} = 2

Answer: The singular values of

A

are

\sigma_1 = 3

and

\sigma_2 = 2

Why It Matters

Singular values are essential in data science and engineering. Principal component analysis (PCA) relies on them to reduce high-dimensional datasets, image compression uses low-rank SVD approximations to store images efficiently, and numerical analysts use the ratio

\sigma_1 / \sigma_p

(the condition number) to assess whether a linear system is numerically stable to solve.

Common Mistakes

Mistake: Confusing singular values with eigenvalues of

A

itself.

Correction: Singular values are the square roots of the eigenvalues of

A^T A

, not of

A

. Eigenvalues of

A

can be negative or complex, but singular values are always non-negative real numbers.