Mathwords logoMathwords

Least-Squares Regression Equation — Formula & Examples

Least-Squares Regression Equation

An equation of a particular form (linear, quadratic, exponential, etc.) that fits a set of paired data as closely as possible. The equation must be chosen so that the sum of the squares of the residuals is made as small as possible.

Scatter plot of black data points with a green best-fit line. Short vertical dashed terracotta segments (residuals) connect each data point to the line.

Key Formula

y^=b0+b1x\hat{y} = b_0 + b_1 x where the slope and intercept are computed as: b1=n∑xiyi−(∑xi)(∑yi)n∑xi2−(∑xi)2b0=yˉ−b1xˉb_1 = \frac{n\sum x_i y_i - \left(\sum x_i\right)\left(\sum y_i\right)}{n\sum x_i^2 - \left(\sum x_i\right)^2} \qquad b_0 = \bar{y} - b_1\bar{x}
Where:
  • y^\hat{y} = The predicted value of the response variable
  • xx = The explanatory (independent) variable
  • b1b_1 = The slope of the regression line
  • b0b_0 = The y-intercept of the regression line
  • nn = The number of data points
  • xi,yix_i, y_i = Individual paired data values
  • xˉ,yˉ\bar{x}, \bar{y} = The means of the x-values and y-values, respectively

Worked Example

Problem: Find the least-squares regression line for the data: (1, 2), (2, 4), (3, 5), (4, 4), (5, 5).
Step 1: Compute the needed sums. With n = 5 data points:
∑xi=15,∑yi=20,∑xiyi=67,∑xi2=55\sum x_i = 15,\quad \sum y_i = 20,\quad \sum x_i y_i = 67,\quad \sum x_i^2 = 55
Step 2: Find the means of x and y.
xˉ=155=3,yˉ=205=4\bar{x} = \frac{15}{5} = 3,\qquad \bar{y} = \frac{20}{5} = 4
Step 3: Calculate the slope using the formula.
b1=5(67)−(15)(20)5(55)−(15)2=335−300275−225=3550=0.7b_1 = \frac{5(67) - (15)(20)}{5(55) - (15)^2} = \frac{335 - 300}{275 - 225} = \frac{35}{50} = 0.7
Step 4: Calculate the y-intercept.
b0=yˉ−b1xˉ=4−0.7(3)=4−2.1=1.9b_0 = \bar{y} - b_1\bar{x} = 4 - 0.7(3) = 4 - 2.1 = 1.9
Step 5: Write the least-squares regression equation.
y^=1.9+0.7x\hat{y} = 1.9 + 0.7x
Answer: The least-squares regression equation is ŷ = 1.9 + 0.7x. For every 1-unit increase in x, the predicted y increases by 0.7.

Another Example

This example focuses on residuals and the sum of squares, demonstrating what 'least squares' actually minimizes rather than just computing the equation.

Problem: Using the regression equation ŷ = 1.9 + 0.7x from the first example, compute all the residuals and verify that the sum of squared residuals is minimized (i.e., show the value that the method minimizes).
Step 1: Compute the predicted value for each data point using ŷ = 1.9 + 0.7x.
y^1=2.6,y^2=3.3,y^3=4.0,y^4=4.7,y^5=5.4\hat{y}_1 = 2.6,\quad \hat{y}_2 = 3.3,\quad \hat{y}_3 = 4.0,\quad \hat{y}_4 = 4.7,\quad \hat{y}_5 = 5.4
Step 2: Calculate each residual: residual = observed y − predicted ŷ.
e_1 = 2 - 2.6 = -0.6,\quad e_2 = 4 - 3.3 = 0.7,\quad e_3 = 5 - 4.0 = 1.0$$ $$e_4 = 4 - 4.7 = -0.7,\quad e_5 = 5 - 5.4 = -0.4
Step 3: Square each residual and add them together.
SSR=(−0.6)2+(0.7)2+(1.0)2+(−0.7)2+(−0.4)2=0.36+0.49+1.00+0.49+0.16=2.50\text{SSR} = (-0.6)^2 + (0.7)^2 + (1.0)^2 + (-0.7)^2 + (-0.4)^2 = 0.36 + 0.49 + 1.00 + 0.49 + 0.16 = 2.50
Step 4: No other line of the form ŷ = b₀ + b₁x produces a sum of squared residuals smaller than 2.50. This is the quantity the least-squares method minimizes.
min⁡∑i=1n(yi−y^i)2=2.50\min \sum_{i=1}^{n}(y_i - \hat{y}_i)^2 = 2.50
Answer: The sum of squared residuals is 2.50. Any other straight line through these data would give a larger value.

Frequently Asked Questions

What is the difference between a least-squares regression equation and a least-squares regression line?
A least-squares regression line is specifically a straight line (ŷ = b₀ + b₁x). A least-squares regression equation is a broader term that includes any model form—linear, quadratic, exponential, and so on—as long as the parameters are chosen to minimize the sum of squared residuals. Every least-squares regression line is a least-squares regression equation, but not every least-squares regression equation is a line.
Why do we square the residuals instead of just adding them?
If you simply add the residuals without squaring, positive and negative errors cancel out, often giving a sum near zero even for a terrible fit. Squaring makes all residuals positive and also penalizes larger errors more heavily, which pushes the best-fit equation closer to the bulk of the data. This approach also leads to clean, closed-form formulas for the slope and intercept.
When should you use a least-squares regression equation?
Use it whenever you have paired data and want to model the relationship between an explanatory variable and a response variable. It is most common in statistics courses for prediction—estimating a y-value from a given x-value. Before fitting, always check a scatterplot to see whether a linear, quadratic, or other model shape is appropriate.

Least-Squares Regression Equation vs. Least-Squares Regression Line

Least-Squares Regression EquationLeast-Squares Regression Line
DefinitionAny equation form (linear, quadratic, exponential, etc.) fit by minimizing squared residualsSpecifically a linear equation ŷ = b₀ + b₁x fit by minimizing squared residuals
Formula formsŷ = b₀ + b₁x, ŷ = ax² + bx + c, ŷ = abˣ, etc.ŷ = b₀ + b₁x only
When to useWhen the scatterplot suggests any particular curve or patternWhen the scatterplot shows a roughly linear trend
ComplexityMay require more parameters and technology to computeSlope and intercept computed with standard formulas or a calculator

Why It Matters

The least-squares regression equation is central to AP Statistics, college introductory statistics, and data science. You use it every time you draw a trend line on a scatterplot or predict values from data. Understanding how it works also builds the foundation for more advanced topics like multiple regression and machine learning models.

Common Mistakes

Mistake: Swapping x and y when computing the slope, which gives the regression of x on y instead of y on x.
Correction: Always place the response variable (the one you want to predict) as y and the explanatory variable as x. The regression of y on x is not the same equation as the regression of x on y.
Mistake: Using the regression equation to predict far outside the range of the original x-values (extrapolation).
Correction: The least-squares equation is only reliable within (or near) the range of x-values in your data. Predicting well beyond that range assumes the same pattern continues, which may not be true.

Related Terms