Choose the response and fit a line

From association to a regression model

Decide which coordinate is the response before fitting. Regression of y on x minimises squared vertical residuals on the usual axes; regression of x on y minimises squared horizontal residuals. Rearranging a fitted line does not generally fit the other response.

Linear regression fits a line to observed paired data. Check the scatter and coefficient before deciding whether a straight-line model is suitable; the usual fitting method is least squares. There are two directions: regression of $y$ on $x$ and regression of $x$ on $y$. A model can support estimation and, with appropriate assumptions, inference about population parameters; a fitted line alone does not establish those assumptions.

The source compares the two lines on the same axes: when $r\approx0$, their directions differ markedly; near $r\approx1$ they almost coincide, also near $r\approx-1$. Two regression directions preserves both schematic comparisons. This describes the same dataset and fixed axis scales; the visual angle also depends on axis scaling. Later source pages supply the residual definitions, coefficients and conditions for choosing the appropriate direction.

Fit the response coordinate: two least-squares problems

For regression of $y$ on $x$, write $y=a+bx$, with $x$ explanatory and $y$ response. Plot $x$ horizontally and $y$ vertically. Each residual is the vertical difference between an observed $y_i$ and its prediction at the same $x_i$:

$$e_i=y_i-(a+bx_i),\qquad S=\sum_{i=1}^{n}e_i^2=\sum_{i=1}^{n}(y_i-a-bx_i)^2.$$

Choose $a,b$ to minimise $S$. It is the sum of squared residuals, not a signed total residual or the sum of perpendicular distances. The source calls $S$ “Total residual”; the precise objective is the sum of squares. It depends on both $a$ and $b$.

$$b=\frac{S_{xy}}{S_{xx}}=\frac{\sum(x-\bar x)(y-\bar y)}{\sum(x-\bar x)^2}=\frac{\sum xy-(\sum x)(\sum y)/n}{\sum x^2-(\sum x)^2/n},$$ $$a=\frac{\sum y_i}{n}-b\frac{\sum x_i}{n}=\bar y-b\bar x,\qquad y-\bar y=b(x-\bar x).$$

Thus $a$ is the $y$-intercept and $b$ the gradient/regression coefficient. For defined $r$, the gradient and correlation have the same sign. The $y$-on-$x$ line passes through $(\bar x,\bar y)$ and requires $S_{xx}>0$. Use the calculator for complete paired data; MF27 formulas are useful when calculator use is not permitted or only sufficient summary information is given.

Fit x as the response

For regression of $x$ on $y$, take $y$ as explanatory and $x$ as response and write $x=c+dy$. On the same conventional axes, its residuals are horizontal:

$$e_i=x_i-(c+dy_i),\qquad S=\sum_{i=1}^{n}(x_i-c-dy_i)^2.$$ $$d=\frac{S_{xy}}{S_{yy}}=\frac{\sum(x-\bar x)(y-\bar y)}{\sum(y-\bar y)^2}=\frac{\sum xy-(\sum x)(\sum y)/n}{\sum y^2-(\sum y)^2/n},$$ $$c=\bar x-d\bar y,\qquad x-\bar x=d(y-\bar y).$$

This line also passes through $(\bar x,\bar y)$ and requires $S_{yy}>0$. The two source residual schematics are retained in Vertical and horizontal residuals. Swapping regression direction changes the quantity minimised; rearranging one fitted equation does not generally produce the other fitted regression. If $d=0$, the reverse line is $x=\bar x$ and cannot be written as a single-valued $y$ function.

The eight-observation example fits both lines, sketches them on the same axes, and uses their intersection to recover the means. Keep full coefficients for this calculation; premature rounding changes the intersection.

Fit both responses with an imperfect dataset

For observations $(1,1),(2,2),(3,2),(4,5)$, use a calculator to fit y on x and x on y. For an observational prediction of x when y=3, compare the reverse fit with rearranging the forward fit. Identify which residual direction each fit minimises; do not derive the fitting formulae.

After your attempt: compare the reasoning

Both sample means are 2.5. The forward line is $y=-1/2+(6/5)x$, while the reverse line is $x=5/6+(2/3)y$. At y=3, the reverse-fit estimate is $17/6\approx2.8333$. Rearranging the forward line gives $35/12\approx2.9167$, a different estimate.

The forward line minimises squared y residuals; the reverse line minimises squared x residuals. For this observational prediction of x from y, use the x-response fit. A controlled-input calibration question can have a different modelling setup, so identify the role of the variables rather than choosing a line from the letter being requested alone.