Overdetermined System of Equations

Q: How do you solve an overdetermined system when no exact solution exists?

When no exact solution exists, the standard approach is the method of least squares. You find the vector x that minimizes the sum of squared errors by solving the normal equations A^T A x = A^T b. This gives the 'best approximate' solution and is widely used in statistics and data fitting.

Overdetermined System of Equations

A linear system of equations in which there are more equations than there are variables. For example, a system with three equations and only two unknowns is overdetermined. Note that an overdetermined system might be either consistent or inconsistent, depending on the equations.

See also

Underdetermined system of equations

Key Formula

A\mathbf{x} = \mathbf{b}, \quad \text{where } A \text{ is an } m \times n \text{ matrix with } m > n

Where:

$A$ = The coefficient matrix with m rows (equations) and n columns (variables)
$\mathbf{x}$ = The column vector of n unknowns
$\mathbf{b}$ = The column vector of m constant terms
$m$ = The number of equations
$n$ = The number of unknowns, where m > n

Worked Example

Problem: Determine whether the following overdetermined system (3 equations, 2 unknowns) is consistent or inconsistent: x + y = 3 2x − y = 0 3x + y = 5

Step 1: Identify that the system is overdetermined: there are 3 equations but only 2 unknowns (x and y), so m = 3 > n = 2.

Step 2: Solve the first two equations. From equation 1, y = 3 − x. Substitute into equation 2.

2x - (3 - x) = 0 \implies 3x - 3 = 0 \implies x = 1

Step 3: Back-substitute to find y.

y = 3 - 1 = 2

Step 4: Check whether the solution (x, y) = (1, 2) satisfies the third equation.

3(1) + 2 = 5 \quad \checkmark

Answer: The system is consistent with the unique solution x = 1, y = 2. All three equations are satisfied.

Another Example

This example shows the more common outcome for overdetermined systems: inconsistency. The first example was consistent, but here the extra equation conflicts with the solution determined by the first two.

Problem: Determine whether this overdetermined system is consistent or inconsistent: x + y = 4 x − y = 2 2x + y = 10

Step 1: The system has 3 equations and 2 unknowns, so it is overdetermined (m = 3 > n = 2).

Step 2: Add equations 1 and 2 to eliminate y.

(x + y) + (x - y) = 4 + 2 \implies 2x = 6 \implies x = 3

Step 3: Substitute x = 3 back into equation 1 to find y.

3 + y = 4 \implies y = 1

Step 4: Check the solution (x, y) = (3, 1) in the third equation.

2(3) + 1 = 7 \neq 10

Answer: The system is inconsistent — no solution exists. The first two equations force (x, y) = (3, 1), but the third equation contradicts this.

Frequently Asked Questions

Can an overdetermined system have a solution?

Yes. An overdetermined system can be consistent if the extra equations are compatible with the others—for instance, if one equation is a linear combination of the rest. However, this is the exception rather than the rule; most overdetermined systems with arbitrary coefficients turn out to be inconsistent.

What is the difference between an overdetermined and an underdetermined system?

An overdetermined system has more equations than unknowns (m > n), so it typically has no solution. An underdetermined system has fewer equations than unknowns (m < n), so it typically has infinitely many solutions. Both can behave differently in special cases: an overdetermined system can be consistent, and an underdetermined system can be inconsistent.

How do you solve an overdetermined system when no exact solution exists?

When no exact solution exists, the standard approach is the method of least squares. You find the vector x that minimizes the sum of squared errors by solving the normal equations A^T A x = A^T b. This gives the 'best approximate' solution and is widely used in statistics and data fitting.

Overdetermined System vs. Underdetermined System

	Overdetermined System	Underdetermined System
Definition	More equations than unknowns (m > n)	Fewer equations than unknowns (m < n)
Typical outcome	No solution (inconsistent)	Infinitely many solutions
Matrix shape	A is tall: more rows than columns	A is wide: more columns than rows
Can it have a unique solution?	Yes, if the extra equations are consistent with the rest	Only in special/degenerate cases (generally no)
Approximate method	Least squares minimizes error	Minimum-norm solution picks the smallest x

Why It Matters

Overdetermined systems arise naturally whenever you collect more data points than you have parameters to fit—such as drawing a best-fit line through many data points in statistics. Understanding them is essential for linear regression, engineering curve fitting, and any situation where measurements outnumber unknowns. Recognizing that a system is overdetermined tells you immediately to check for consistency before attempting a unique solution.

Common Mistakes

Mistake: Assuming an overdetermined system can never have a solution.

Correction: An overdetermined system is often inconsistent, but not always. If the extra equations are dependent on (or compatible with) the others, a solution exists. Always check by solving and substituting into every equation.

Mistake: Trying to find an exact solution by ignoring some equations.

Correction: Dropping equations changes the system entirely. If no exact solution exists, use the least-squares method to find the best approximation that accounts for all equations simultaneously.

Related Terms

Linear System of Equations — General category that includes overdetermined systems
Underdetermined System of Equations — Opposite case: fewer equations than unknowns
Consistent System of Equations — An overdetermined system that does have a solution
Inconsistent System of Equations — The typical outcome when a system is overdetermined
Variable — The unknowns whose count defines overdetermined status
Equation — The constraints whose count exceeds the number of variables