Last updated: 2026-06-23

Definition

Simple linear regression models the relationship between an independent variable x and a dependent variable y by fitting a straight line of the form

$\hat{y} = a + b\,x$

where b is the slope (the average change in y per unit change in x) and a is the intercept (the predicted value of y when x = 0). The fitted value ŷ is an estimate of y for a given x; the difference between an observed value yᵢ and its fitted value, $e_i = y_i - \hat{y}_i$, is called the residual.

Least squares

Many lines could be drawn through a cloud of points. The method of least squares selects the one that minimizes the sum of squared residuals,

$\sum_{i=1}^{n} e_i^2 = \sum_{i=1}^{n}\bigl(y_i - a - b\,x_i\bigr)^2.$

Squaring penalizes large deviations more than small ones and prevents positive and negative residuals from cancelling. Minimizing this sum has a closed-form solution, so no iteration is required.

Formulas

Let $\bar{x}$ and $\bar{y}$ be the means of x and y. The least-squares slope and intercept are

$b = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2}, \qquad a = \bar{y} - b\,\bar{x}.$

The numerator of b measures how x and y co-vary; the denominator is the spread of x. The intercept formula guarantees that the fitted line passes through the point of means $(\bar{x}, \bar{y})$.

The coefficient of determination

The coefficient of determination, $R^2$, reports the fraction of the variance in y that the fitted line explains:

$R^2 = 1 - \frac{\sum_i (y_i - \hat{y}_i)^2}{\sum_i (y_i - \bar{y})^2}.$

It ranges from 0 (the line explains none of the variation) to 1 (the line passes exactly through every point). In simple linear regression, $R^2$ equals the square of the Pearson correlation coefficient r, so a strong correlation implies a high $R^2$. See the Pearson Correlation Coefficient Calculator calculator for the related measure of association.

Worked example

Using the default dataset, X = [1, 2, 3, 4, 5] and Y = [2, 4, 5, 4, 5], the means are $\bar{x} = 3$ and $\bar{y} = 4$. Computing the deviation sums gives $S_{xx} = \sum(x_i - \bar{x})^2 = 10$, $S_{xy} = \sum(x_i - \bar{x})(y_i - \bar{y}) = 6$, and $S_{yy} = \sum(y_i - \bar{y})^2 = 6$.

The fitted line ŷ = 2.2 + 0.6·x rises by 0.6 in y for each unit of x, and accounts for about 60% of the variation in y.

Assumptions and limits

The least-squares line is most reliable when several conditions hold:

• Linearity. A straight line should be a reasonable summary of the pattern. If the points curve, a linear fit can still produce a moderate $R^2$ while systematically missing the trend; inspect the scatter plot before trusting the slope.
• Outliers. Because residuals are squared, a single distant point can pull the line toward it and distort both the slope and $R^2$. Examine extreme points before drawing conclusions.
• Range of prediction. Extrapolating far outside the observed range of x is risky — the linear relationship may not hold there, and the intercept is only interpretable when x = 0 is near the data.
• Correlation is not causation. A good fit shows that x and y move together; it does not establish that x causes y. A lurking third variable may drive both. Standardizing variables with a Z-Score Calculator can help compare effects, but causal claims require a controlled study.

These caveats notwithstanding, simple linear regression remains one of the most widely used tools in statistics, providing a transparent, closed-form summary of how one variable depends on another.