Porisma

Correlation and linear regression

← Back to learning units · Complete chapter reading and print view · Open reference notes separately

By the end, you can

  • Use scatterplots before fitting a line
  • Interpret correlation without claiming causation
  • Distinguish interpolation, extrapolation and transformed models

about 40 minutes for the lesson · practise at your own pace.

The central idea

Correlation describes linear association, while a regression line predicts a response from an explanatory variable. The least-squares line minimises squared vertical residuals. A strong association does not establish causation. Prediction beyond the observed input range is extrapolation; transformations can reveal a relationship hidden in the original coordinates.

Begin with paired observations

The notes ask about links between GenAI use and examination scores, and between temperature and rainfall intensity. These are questions to investigate, not established conclusions. In the source’s controlled experiment, vary the slope angle through 10∘,20∘,30∘,40∘,50∘10^\circ,20^\circ,30^\circ,40^\circ,50^\circ and record a ball’s travel time. The angle is the independent (explanatory) variable, and the travel time is the dependent (response) variable.

When control is impractical, the source proposes observing temperature and sea level each day over ten years. This is observational data collection. The notes use “natural experiments” broadly here; an observational study is not automatically a natural experiment with an externally imposed change. Controlled studies are possible in some settings even in fields mentioned by the source; identify what is actually controlled in the particular investigation.

Bivariate data consist of ordered pairs (x1,y1),…,(xn,yn)(x_1,y_1),\ldots,(x_n,y_n), such as each person’s age and salary. Each pair must refer to the same observation. The explanatory variable supplies the basis for prediction or study; the response is the outcome observed or predicted. In an observational study the explanatory variable need not be manipulated. Calling a variable “independent” in this role does not assert statistical independence of X and Y.

Try the five source situations before opening their working.

Inspect a scatter diagram

A scatter diagram plots the paired numerical observations, usually with the explanatory variable on the horizontal x-axis and the response on the vertical y-axis. The notes also call it a scatter plot, scatter graph or scatter chart. Inspect:

  • Shape: approximately linear, curved (such as the source’s inverted U), or another pattern (such as its loop).
  • Direction for a linear pattern: positive/direct means y tends to rise with x; negative/inverse means y tends to fall with x. “Inverse” here describes direction, not a proved reciprocal formula y=k/xy=k/x.
  • Strength of the linear pattern: tight clustering about a line is stronger than a widely scattered cloud. A strong curved pattern need not have strong linear correlation.

All eight schematic panels from the opening notes are reconstructed in the accompanying figure: three shape examples, two direction examples and three strength examples. They are illustrations, not measured numerical data.

The source uses numerical variables such as height, mass, rates and counts. It contrasts beauty and happiness as not directly numerical. Such concepts can be studied using explicitly defined numerical scores, but a score’s meaning and measurement quality need justification; merely assigning category numbers does not establish a quantitative scale.

For the linear correlation studied here, the source’s sequence is sketch the scatter diagram, then find the correlation coefficient. Plotting first can reveal curvature or unusual observations hidden by a single summary number.

Transfer a calculator plot to a clear sketch

Retain all five source pointers:

  1. Check the window scale: the origin may not be visible.
  2. Label both axes and put the explanatory variable horizontally, response vertically.
  3. Mark the minimum and maximum x and y values.
  4. Match the number of observations with the number of plotted dots or crosses. Coincident pairs can overlap, so record their multiplicity if needed.
  5. Preserve relative positions accurately. Graph paper is unnecessary unless requested.

Example 1 retains every data pair and the complete six-step calculator route before describing the observed pattern.

Eight scatter patterns

Read shape, direction and scatter

Compare the patterns below.

Linear relationship

6 points

Linear relationship. Inspect the shape as well as the direction.

Curved: inverted U

9 points

Curved: inverted U. Inspect the shape as well as the direction.

Curved: loop

8 points

Curved: loop. Inspect the shape as well as the direction.

Positive linear

5 points

Positive linear. Inspect the shape as well as the direction.

Negative linear

5 points

Negative linear. Inspect the shape as well as the direction.

Strong negative linear

4 points

Strong negative linear. Inspect the shape as well as the direction.

Strong positive linear

6 points

Strong positive linear. Inspect the shape as well as the direction.

Weak/no clear linear

13 points

Weak/no clear linear. Inspect the shape as well as the direction.
  • Schematic illustrations: positions are not measured observations. No numerical data or calculated correlation is implied.
  • Look for shape before describing direction or strength.
  • A clear curved relationship can have little or no linear correlation.
Eight schematic panels, retaining source point counts: positive line 6 points, inverted-U curve 9, loop 8, positive line 5, negative line 5, strong negative 4, strong positive 6, and weak/no clear linear cloud 13. A curved or loop relationship can be strong without being linear. Positions illustrate patterns, not measured data.
Open original diagram (new tab)

Five coefficient cases

Compare five correlation cases

Compare the patterns below.

Perfect negative · r = −1

3 points

Perfect negative · r = −1. Inspect the shape as well as the direction.

Negative · −1 < r < 0

3 points

Negative · −1 < r < 0. Inspect the shape as well as the direction.

Zero linear correlation · r = 0

8 points

Zero linear correlation · r = 0. Inspect the shape as well as the direction.

Positive · 0 < r < 1

3 points

Positive · 0 < r < 1. Inspect the shape as well as the direction.

Perfect positive · r = +1

3 points

Perfect positive · r = +1. Inspect the shape as well as the direction.
  • Schematic illustrations: positions are not measured observations. No numerical data or calculated correlation is implied.
  • The coefficient labels retain the roles stated in the source, not values computed from these schematic positions.
  • Zero linear correlation does not imply independence or no relationship.
Five schematic panels reproduce the source roles: perfect negative, imperfect negative, zero linear correlation with a curved arrangement, imperfect positive, perfect positive. The imperfect panels show scatter around the drawn line. These are schematic, not supplied numerical observations.
Open original diagram (new tab)

Causation and equal-coefficient patterns

Read the scatter as well as the coefficient

Compare the patterns below.

Left and right foot size

5 points

The rising pattern shows association. It does not establish that the size of one foot causes the size of the other.

Linear-looking scatter · source r = 0.86

11 points

The points show an approximately linear pattern with scatter. The coefficient is the source report for its original data set.

Curved pattern · source r = 0.86

11 points

A similar reported coefficient can accompany a curved pattern. A straight-line description misses this shape.

One unusual observation · source r = 0.86

11 points

Most points lie close to a line, with one unusual observation. Inspect the whole pattern before interpreting the coefficient.
  • Ringed point: Unusual observation in this schematic.
  • Strong correlation alone does not establish causation. In the foot-size example, shared influences such as genes can explain the association.
  • The source reports r = 0.86 for each of the three original data sets below the foot-size illustration. These reconstructed positions are schematic; they are not the raw data used to calculate that coefficient.
  • A coefficient alone does not reveal curvature or an unusual observation. Inspect the diagram before interpreting a linear relationship.
A five-point rising foot-size schematic and three eleven-point source-pattern schematics: scattered approximately linear, curved, and an otherwise close linear pattern with one unusual observation. The original source reports r0.86 for each of the three, but supplies no raw data; these reconstructed positions are illustrations rather than numerical datasets.
Open original diagram (new tab)

Outlier in a rising scatter

Inspect how an outlier changes the pattern

Compare the patterns below.

Source outlier in a rising pattern

14 points

The other points lie closer to a rising straight-line pattern.
  • Ringed point: Source outlier (2, 7).
  • The source identifies (2, 7) exactly. Other point positions in this reconstruction are schematic; no complete numerical data set is supplied.
  • In this example, the source states that removing (2, 7) raises the positive correlation because the remaining pattern is closer to a rising straight line.
  • An outlier can increase or decrease a correlation coefficient. Investigate the observation and context before deciding whether exclusion is justified.
Fourteen-point source scatter reconstructed schematically, with the explicitly stated outlier (2,7) above the otherwise rising pattern. Removing it raises the positive correlation in this source example. Remaining coordinates are not supplied as a numerical dataset.
Open original diagram (new tab)

Two regression directions

Compare the fitted response on the same x-horizontal and y-vertical axes. These source-role sketches preserve illustrative points and lines, not numerical observations or regressions calculated from the drawing.

Weak linear association

  • Solid: y on x
  • Dashed: x on y
In the source's near-zero-correlation sketch, the two line directions differ greatly.

Strong linear association

  • Solid: y on x
  • Dashed: x on y
In the source's near-perfect positive-correlation sketch, the two lines nearly coincide. The notes also make this comparison near r = −1.

Keep the two response models distinct

The y-on-x fit minimises squared vertical residuals; the x-on-y fit minimises squared horizontal residuals on these axes. Rearranging one fitted equation does not generally produce the other fitted regression.

y=a+bxx=c+dy\begin{aligned}y&=a+bx\\x&=c+dy\end{aligned}

Both fitted lines pass through the means

For paired data with both variables varying, both least-squares regressions pass through this mean point. Use the actual observations or fitted equations to find it; the schematic positions supply no numerical mean.

(xˉ,yˉ)(\bar x,\bar y)

At exactly zero linear correlation

When both variables vary and r is zero, both regression coefficients are zero. On the same axes, y-on-x is horizontal and x-on-y is vertical. The vertical line cannot be rewritten as a single-valued function y of x.

y=yˉx=xˉ\begin{aligned}y&=\bar y\\x&=\bar x\end{aligned}
  • Use the same axes and scale when comparing the two directions. A drawn angle is not a scale-independent measure of correlation; rescaling an axis changes the angle. Do not read a numerical r from these sketches.
  • With perfect linear association and both variables varying, the fitted lines coincide. A constant variable makes the usual correlation coefficient undefined, rather than providing a zero-correlation example.
  • Choosing a regression for prediction still depends on the prescribed explanatory role. Use the linked method to distinguish direct prediction from inverse calibration.
  • Zero linear correlation does not rule out a nonlinear relationship. Inspect the scatter before accepting a line.

Compare with the source direction sketches

Expand diagram

Use the scrollbars or arrow keys to inspect details.

Two source-role schematics on common x-horizontal and y-vertical axes: near-zero correlation gives nearly horizontal y-on-x and nearly vertical x-on-y lines; near-perfect positive correlation gives almost coincident lines. The same convergence applies near perfect negative correlation. Positions and lines illustrate roles, not numerical fits.
Two source-role schematics on common x-horizontal and y-vertical axes: near-zero correlation gives nearly horizontal y-on-x and nearly vertical x-on-y lines; near-perfect positive correlation gives almost coincident lines. The same convergence applies near perfect negative correlation. Positions and lines illustrate roles, not numerical fits.
Two source-role schematics on common x-horizontal and y-vertical axes: near-zero correlation gives nearly horizontal y-on-x and nearly vertical x-on-y lines; near-perfect positive correlation gives almost coincident lines. The same convergence applies near perfect negative correlation. Positions and lines illustrate roles, not numerical fits.
Open original diagram (new tab)

Vertical and horizontal residuals

Choose the response before measuring a residual. Both sketches keep x horizontal and y vertical. Their point positions and lines are schematic; they are not numerical data or fits calculated from those positions.

y on x: vertical residuals

Ten schematic source points and a retained illustrative line. Four dashed vertical segments mark source residual roles; they are not perpendicular distances.

Predict y from x

Keep the observed x coordinate fixed. The line supplies the predicted y response at that same input.

y=a+bxy=a+bx

Measure the vertical residual

The residual is observed y minus predicted y. It is signed; the dashed segment shows its magnitude along the response coordinate.

ei=yi−y^i=yi−(a+bxi)\begin{aligned}e_i&=y_i-\hat y_i\\&=y_i-(a+bx_i)\end{aligned}

Minimise squared vertical residuals

Choose a and b to minimise this sum over every observation. Squaring prevents positive and negative residuals from cancelling. The drawing marks four residual roles, not only four terms in the sum.

Sy=∑i=1nei2S_y=\sum_{i=1}^n e_i^2

Compare each observation with its prediction.

Dashed vertical residual from the ringed observation to its prediction.
  • Ringed point: e₁.

e₁: below its prediction

The ringed observation lies below its prediction, so observed y minus predicted y is negative.

e1=y1−y^1<0e_{1}=y_{1}-\hat y_{1}<0
Dashed vertical residual from the ringed observation to its prediction.
  • Ringed point: e₂.

e₂: above its prediction

The ringed observation lies above its prediction, so observed y minus predicted y is positive.

e2=y2−y^2>0e_{2}=y_{2}-\hat y_{2}>0
Dashed vertical residual from the ringed observation to its prediction.
  • Ringed point: e₃.

e₃: below its prediction

The ringed observation lies below its prediction, so observed y minus predicted y is negative.

e3=y3−y^3<0e_{3}=y_{3}-\hat y_{3}<0
Dashed vertical residual from the ringed observation to its prediction.
  • Ringed point: eₙ.

eₙ: below its prediction

The ringed observation lies below its prediction, so observed y minus predicted y is negative.

en=yn−y^n<0e_{n}=y_{n}-\hat y_{n}<0

x on y: horizontal residuals

Ten schematic source points and a retained illustrative line. Four dashed horizontal segments mark source residual roles; they are not perpendicular distances.

Predict x from y

Keep the observed y coordinate fixed. The line supplies the predicted x response at that same input.

x=c+dyx=c+dy

Measure the horizontal residual

The residual is observed x minus predicted x. It is signed; the dashed segment shows its magnitude along the response coordinate.

ei=xi−x^i=xi−(c+dyi)\begin{aligned}e_i&=x_i-\hat x_i\\&=x_i-(c+dy_i)\end{aligned}

Minimise squared horizontal residuals

Choose c and d to minimise this sum over every observation. Squaring prevents positive and negative residuals from cancelling. The drawing marks four residual roles, not only four terms in the sum.

Sx=∑i=1nei2S_x=\sum_{i=1}^n e_i^2

Compare each observation with its prediction.

Dashed horizontal residual from the ringed observation to its prediction.
  • Ringed point: e₁.

e₁: right of its prediction

The ringed observation lies right of its prediction, so observed x minus predicted x is positive.

e1=x1−x^1>0e_{1}=x_{1}-\hat x_{1}>0
Dashed horizontal residual from the ringed observation to its prediction.
  • Ringed point: e₂.

e₂: left of its prediction

The ringed observation lies left of its prediction, so observed x minus predicted x is negative.

e2=x2−x^2<0e_{2}=x_{2}-\hat x_{2}<0
Dashed horizontal residual from the ringed observation to its prediction.
  • Ringed point: e₃.

e₃: right of its prediction

The ringed observation lies right of its prediction, so observed x minus predicted x is positive.

e3=x3−x^3>0e_{3}=x_{3}-\hat x_{3}>0
Dashed horizontal residual from the ringed observation to its prediction.
  • Ringed point: eₙ.

eₙ: right of its prediction

The ringed observation lies right of its prediction, so observed x minus predicted x is positive.

en=xn−x^n>0e_{n}=x_{n}-\hat x_{n}>0
  • Changing the response changes the least-squares problem. Rearranging one fitted equation does not generally give the other fitted regression.
  • Measure along the response coordinate, not perpendicular to the line. A residual may be positive, negative or zero; its square is nonnegative.
  • The two source panels have distinct schematic observations. Do not treat them as numerical fits of one shared dataset, or calculate coefficients or residual totals from their drawing coordinates.
  • The two sums measure errors in different response coordinates and may have different units. Compare candidate fits for the same response on the same observations; do not rank y-on-x against x-on-y by their raw residual totals.

Compare with the source residual sketches

Expand diagram

Use the scrollbars or arrow keys to inspect details.

Two distinct source residual schematics: ten observations with vertical differences for y-on-x, and ten observations with horizontal differences for x-on-y. Four residual roles e1,e2,e3, en marked in each. Dashed distances are measured along the response coordinate, not perpendicular to the line. Positions are schematic; formulas are in the accompanying text.
Two distinct source residual schematics: ten observations with vertical differences for y-on-x, and ten observations with horizontal differences for x-on-y. Four residual roles e1,e2,e3, en marked in each. Dashed distances are measured along the response coordinate, not perpendicular to the line. Positions are schematic; formulas are in the accompanying text.
Two distinct source residual schematics: ten observations with vertical differences for y-on-x, and ten observations with horizontal differences for x-on-y. Four residual roles e1,e2,e3, en marked in each. Dashed distances are measured along the response coordinate, not perpendicular to the line. Positions are schematic; formulas are in the accompanying text.
Open original diagram (new tab)

Interpolation and extrapolation

For direct prediction of a response y from an explanatory input x, compare the input with its observed range. This source illustration is schematic and supplies no numerical observations.

The dashed guides mark the smallest and largest observed inputs. The coloured bar spans this input range.

Below the observed inputs

Extrapolation. The explanatory input is smaller than every input observed in the sample.

x0<xmin⁡x_0<x_{\min}

Within the observed input range

Interpolation. The input lies between the smallest and largest observed inputs, including the limits.

xmin⁡≤x0≤xmax⁡x_{\min}\le x_0\le x_{\max}

Above the observed inputs

Extrapolation. The explanatory input is larger than every input observed in the sample.

x0>xmax⁡x_0>x_{\max}
  • Check the explanatory-variable range. For a y-on-x prediction, compare the known x input with its observed x range, rather than with the response y range.
  • Being inside the range does not guarantee accuracy. Check that the scatter supports the model; two observations alone cannot establish an underlying linear relationship.
  • Keep the population and measurement context relevant. A strong correlation within a sample does not establish that the relationship continues outside the observed inputs.

Compare with the source range sketch

Expand diagram

Use the scrollbars or arrow keys to inspect details.

Twelve-point source-role rising scatter between two dashed explanatory-variable limits. Predictions inside these limits are interpolation; predictions on either side are extrapolation. Positions are schematic and no numerical observations are supplied. The plotted horizontal limits describe input range, not response range.
Twelve-point source-role rising scatter between two dashed explanatory-variable limits. Predictions inside these limits are interpolation; predictions on either side are extrapolation. Positions are schematic and no numerical observations are supplied. The plotted horizontal limits describe input range, not response range.
Twelve-point source-role rising scatter between two dashed explanatory-variable limits. Predictions inside these limits are interpolation; predictions on either side are extrapolation. Positions are schematic and no numerical observations are supplied. The plotted horizontal limits describe input range, not response range.
Open original diagram (new tab)

Explore

Use the chemical-and-bacteria data in Review 2. Fit y on x and then x on y. Draw the residuals in the response direction for each fit: vertical for y on x, horizontal for x on y. Explain why each line minimises a different sum of squared discrepancies, so rearranging one fitted equation generally does not produce the other.

For the comparison requested in the question, refit without the suspect observation and inspect both the scatter and correlation again. In an investigation, excluding an observation needs a reason beyond improving the coefficient. Explain why correlation alone cannot establish a suitable model or a causal explanation, and check whether a proposed prediction stays inside the observed range.

Worked example

For (1,1),(2,2),(3,2),(4,5)(1,1),(2,2),(3,2),(4,5), both sample means are 2.5. A calculator gives y=−0.5+1.2xy=-0.5+1.2x for y on x and x=5/6+(2/3)yx=5/6+(2/3)y for x on y, with r≈0.8944r\approx0.8944. The first minimises squared y residuals and the second squared x residuals. At y=3, the reverse fit estimates x as 17/617/6, whereas rearranging the forward line gives 35/1235/12. The different estimates reflect different fitting objectives. These are observational predictions, with the response chosen for the modelling task.

Inspect the scatter, residual direction and modelling setup. A large correlation does not establish causation or justify extrapolation, and rearranging a fit does not generally refit the other response.

Check your understanding

An observational dataset has xˉ=5\bar x=5, yˉ=20\bar y=20, Sxx=20S_{xx}=20, Syy=160S_{yy}=160 and Sxy=40S_{xy}=40, where S denotes centred sums. Observed x values span 2 to 8 and y values span 12 to 28. Find both regression lines. Choose the appropriate fitted line to estimate y when x=7 and x when y=24. Compare the latter estimate with rearranging the other line, and assess a prediction at x=20.

Mastery learning objectives

At the end of this chapter, I should be able to:

Understand, in the context of the scatter diagram, how to:

Identify the independent and dependent variables.

Describe the nature of correlation between two variables.

Interpret the values of the correlation coefficient rr.

Understand, in the context of regression lines, how to:

Model the relationship between two variables by fitting a linear equation to observed data.

Choose the appropriate regression line.

Distinguish between interpolation and extrapolation.

From given data, use the graphing calculator where possible, to:

Obtain required values such as xˉ\bar x, yˉ\bar y, ∑x\sum x and ∑y\sum y.

Find the value of rr.

Sketch a scatter diagram.

Obtain the regression equation and sketch its line on the scatter diagram.

Explain whether an estimate is reliable.

Calculate the sum of squared residuals (the source calls this the “sum of least squares”).

Compare and explain the appropriateness of different models.

Choose a practice question

The source groups practice by skill. Its reference labels are retained below; each link opens the full exercise so you can locate the named part.

SkillLecture examplesDiscussion questionsReview follow-up
Calculate the r-value2(ii), 5(i), 8(i), 9(i), 10(ii)(iii), 11(ii)1 (1st part), 2(i), 3(iv), 4(b)(ii), 6(a)(ii), 8(f)1, 2(i), 3(ii), 4(i)(iv), 5(iii), 6(ii), 7(ii)(iii), 8(b)(ii), 9(ii)
Sketch a scatter diagram1 (1st part), 2(i), 3(iii), 10(i), 11(i)2(i), 3(i), 4(a) & (b)(i), 5(i), 6(a)(i), 7(ii), 8(d)1, 2(ii), 3(i), 4(i), 5(i), 6(i), 7(ii), 8(a) & (b)(i), 9(i)
Comment on the relationship using r, the scatter diagram and/or context1 (2nd part), 2(i), 10(i)(ii), 11(iii)1 (last part), 3(iv), 5(ii)(iii), 6(a)(iii), 7(ii), 8(a)1(i), 3(i)(a)(b), 4(i), 5(ii), 6(ii), 7(iv), 8(b)(ii)
Obtain a regression equation3, 5(ii), 8(ii)2(iii), 3(iii), 4(b)(iii), 5(iv), 7(iii), 8(f)1(ii), 2(i), 3(iii), 4(iii), 6(iii)(iv), 7(iii), 8(b)(iv), 9(ii)(iii)
Determine a variable using an appropriate regression and/or assess prediction validity7, 8(iv), 11(iv)3(v)(vi), 4(iv)(v), 6(a)(iv), 7(iii), 8(g)2(iv), 3(iv)(vi), 4 (1st part), 5(iv), 6(v), 7(v), 8(b)(v)
Identify outliers and perform necessary calculations2(iii), 5(iii)8(c)2(iii)(iv), 3
Explain in context why strong correlation does not imply causation—2(ii), 6(a)(v)4(ii)
Use the fact that a regression line passes through the mean point3 (last part), 4, 8(iii)3(ii), 6(b), 7(i)4(v), 7(i)
Use invariance of r under changes of measurement scale9(ii)7(v), 8(b)3(v), 8(b)(iii)
Use the sum of squared residuals68(e)—

Source-map notes: the prediction row prints Discussion “4(iv)(v)”; these refer to the age/earnings question’s (b)(iv)(v).

The printed Review “4 (1st part)” is preserved as source shorthand; open its full prompt to identify the requested task.

The final row names Discussion8(e); Discussion9(i)(b)–(d), (ii)–(iii) also directly practises residuals and squared-error comparison. This last suggestion is an authored extension to the printed map.

Positive rescaling leaves r unchanged; multiplying only one variable by a negative scale reverses its sign.

Practice questions

Practice section 1 · Questions 1–6

Identify explanatory and response variables · Sketch the eleven insulin and time observations · +4 more

Practice section 2 · Questions 7–12

Compare two scatters and fit both regression directions · Compare two models using squared residuals · +4 more

Practice section 3 · Questions 13–18

Compare linear and squared-speed models of drag · Calculate correlation from six summary values · +4 more

Practice section 4 · Questions 19–24

Compare pollution models and audit a damaged regression fragment · Recover a missing price index and choose the prediction direction · +4 more

Practice section 5 · Questions 25–30

Refit a logarithmic model after finding a recording error · Transform PSI and recover an additional observation · +4 more

Practice section 6 · Question 31

Compare a linear trend with a transformed quadratic

Ready to try this on your own?

For “Correlation and linear regression”, try a question without the working and explain your method. This records your confidence, saved in this browser only.

Ready to practise?Chapter practice