Sampling and the sample mean

Statistics, parameters, estimators and estimates

A parameter, such as population mean $\mu$, variance $\sigma^2$ or coffee-drinking proportion $p$, is fixed for a specified population. A statistic is a function of a random sample: mean, median, mode, standard deviation and proportion are examples.

In the lecture’s coffee example, if $X$ counts morning coffee drinkers in a sample of size $n$, the random sample proportion is $P_s=X/n$; after collecting data its realised value is $p_s=x/n$. Distinguish the random rule from its observed value.

An estimator $T$ is a statistic used to estimate a parameter $\theta$; an estimate $t$ is its realised numerical value.

Unbiasedness means $E(T)=\theta$ over repeated sampling, not $T=\theta$ in every sample. The teacher suggests following the short mean proof first and self-reading the longer variance proof; both are retained below.

Why the mean and variance estimates are unbiased

For iid observations with common finite mean $\mu$ and variance $\sigma^2$, let $\bar X=\frac1n\sum_{i=1}^nX_i$. By linearity,

$E(\bar X)=\frac1n E(X_1+\cdots+X_n)=\frac1n[E(X_1)+\cdots+E(X_n)]=\frac{n\mu}{n}=\mu.$

This expectation calculation only needs the common mean. Independence is needed when dropping covariances in the variance calculation:

$\operatorname{Var}(\bar X)=\frac1{n^2}\operatorname{Var}(X_1+\cdots+X_n)=\frac1{n^2}\sum_{i=1}^n\operatorname{Var}(X_i)=\frac{n\sigma^2}{n^2}=\frac{\sigma^2}{n}.$

The standard error is $\sigma/\sqrt n$. For $n>1$ and positive variance, this variance is smaller than the individual variance; at $n=1$ it is equal.

For dependent observations, covariance terms must be included. An exact finite-population sample without replacement may require a finite-population correction; do not silently replace that sampling design with iid draws.

For $n>1$, define

$S^2=\frac1{n-1}\sum_i(X_i-\bar X)^2=\frac1{n-1}\left[\sum_i X_i^2-\frac{(\sum_iX_i)^2}{n}\right].$

Expand the centred sum:

$\sum_i(X_i-\bar X)^2=\sum_iX_i^2-2\bar X\sum_iX_i+n\bar X^2=\sum_iX_i^2-n\bar X^2,$

because $\sum_iX_i=n\bar X$. Here $\bar X$ is common to every summand, but it is still a random variable; the source’s phrase “$\bar X$ is a constant” is only meaningful with respect to the summation index. We do not pull it out of an expectation as a deterministic quantity.

Using $E(X_i^2)=\sigma^2+\mu^2$ and $E(\bar X^2)=\operatorname{Var}(\bar X)+[E(\bar X)]^2=\sigma^2/n+\mu^2$,

$E(S^2)=\frac{n(\sigma^2+\mu^2)-n(\sigma^2/n+\mu^2)}{n-1}=\frac{(n-1)\sigma^2}{n-1}=\sigma^2.$

Thus the observed $s^2$ is an unbiased variance estimate. This does not say that $S=\sqrt{S^2}$ is generally unbiased for $\sigma$.

Formula and calculator notation tables

Choose the case that matches the supplied data. Calculate the mean and variance separately, then check which denominator the calculator uses.

Given data: $\sum x,\sum x^2$ and $n$

• Mean estimate: $\bar x=\sum x/n$
• Unbiased variance estimate: $s^2=[\sum x^2-(\sum x)^2/n]/(n-1)$

Given data: $\sum(x-a),\sum(x-a)^2$ and $n$

• Mean estimate: $\bar x=a+\sum(x-a)/n$
• Unbiased variance estimate: $s^2=[\sum(x-a)^2-(\sum(x-a))^2/n]/(n-1)$

Given data: Variance computed with denominator $n$

• Mean estimate: Use the observed mean
• Unbiased variance estimate: Multiply that variance by $n/(n-1)$

Quantity: Population parameter

• Mean notation: $\mu$
• Variance notation: $\sigma^2$

Quantity: Observed sample summaries

• Mean notation: $\bar x$
• Variance notation: Source “sample variance”: $\sum(x-\bar x)^2/n$; calculator $\sigma_x^2$

Quantity: Unbiased population estimates

• Mean notation: $\bar x$
• Variance notation: $s^2=\sum(x-\bar x)^2/(n-1)$; calculator $S_x^2$

“Sample variance” is used with different denominator conventions in different texts. This source uses the denominator-$n$ version in its MF27 comparison, then multiplies by $n/(n-1)$. Check the formula rather than inferring it from the name.

For nonconstant data, calculator $S_x$ is larger than $\sigma_x$ because its squared value has denominator $n-1$; both are zero for constant data.

Square the stored $S_x$ to obtain $s^2$ and avoid premature rounding.

Raw values go in L1; a frequency table also requires its frequencies in L2, as Examples 2 and 3 show.

Why shifting simplifies the arithmetic

The lecture contrasts $\sum x=1000005$ with $\sum(x-10000)=5$ for $n=100$. A shift changes the mean but not the centred sum of squares. First,

$\frac{\sum(x-a)}n+a=\frac{\sum x-na}n+a=\frac{\sum x}n.$

Next, expand both terms:

$$\begin{aligned} \sum(x-a)^2-\frac{[\sum(x-a)]^2}{n} &=\sum x^2-2a\sum x+na^2-\frac{(\sum x-na)^2}{n}\\ &=\sum x^2-2a\sum x+na^2-\frac{(\sum x)^2}{n}+2a\sum x-na^2\\ &=\sum x^2-\frac{(\sum x)^2}{n}. \end{aligned}$$

Dividing by $n-1$ proves the shifted variance formula. Do not add the shift back to a variance.

Calculate, then explain what the estimate means

For ten observations, $\sum(x-50)=20$ and $\sum(x-50)^2=130$. Find estimates of the population mean and variance. Explain which estimate needs the shift restored and why the variance estimate uses 9.

After your attempt: compare the reasoning

The mean estimate is $50+20/10=52$. The unbiased variance estimate is $[130-20^2/10]/9=10$. Shifting changes the mean but leaves centred deviations unchanged. The deviations from the sample mean must sum to zero, so only nine can vary freely; this motivates the denominator $n-1$. The estimate 10 is not a guarantee that the population variance equals 10.

Sampling distributions: exact normal and CLT cases

Start by identifying the random quantity: one observation, the sample mean, or the total. Then decide whether normality is exact or justified as an approximation.

A sampling distribution describes the values of a statistic across repeated samples of the same size. It is not the distribution of the observations within one sample. The teacher suggests a spreadsheet illustration; the comparison figure below shows the same change in shape and spread mathematically.

For iid $X_i\sim N(\mu,\sigma^2)$, the sample mean is exactly $\bar X\sim N(\mu,\sigma^2/n)$ and the total is exactly $T=\sum X_i\sim N(n\mu,n\sigma^2)$ for any positive integer $n$.

Normality is preserved under independent linear combinations; no large-sample argument is needed.

For iid observations with finite variance $0<\sigma^2<\infty$, the central limit theorem states that

$\frac{\bar X-\mu}{\sigma/\sqrt n}\xrightarrow{d}N(0,1)\quad\text{as }n\to\infty.$

For a sufficiently large sample, write $\bar X\approx N(\mu,\sigma^2/n)$ by CLT, or equivalently $T\approx N(n\mu,n\sigma^2)$.

This is a limiting distribution result used for approximate probabilities and inference, as the source’s “Limits” callout explains.

The source suggests $n\ge30$ as a working guideline; it is not a universal guarantee for highly skewed or heavy-tailed distributions. The quality depends on the population and on the event being approximated, especially a tail.

If $X$ is discrete, $\bar X$ remains discrete even when a continuous normal approximation is useful.

Increasing $n$ changes the sampling distribution, not the individual-observation distribution.

For small $n$ and an unknown population distribution, mean and variance alone do not determine an exact normal probability. State and justify any extra normality assumption rather than treating it as a fact; a near-normal population may support an approximation.

Larger $n$ gives asymptotic convergence, not a promise that every finite-sample tail error decreases monotonically.

Decide which distribution you are describing

Compare two sets of independent observations: one from a normal population and one from a strongly right-skewed population with finite variance. Both populations have mean 40 and variance 100. For each population, describe what can be said about sample means based on 4 observations and on 100 observations. Distinguish a distribution’s moments from a justified normal model.

After your attempt: compare the models

In both cases $E(\bar X)=40$. The variances are 25 for $n=4$ and 1 for $n=100$.

For the normal population, the means are exactly $N(40,25)$ and $N(40,1)$.

For the skewed population, four observations do not justify assuming a normal mean.

At 100 observations a CLT approximation may be appropriate, but adequate sample size depends on the skewness and the probability being estimated.

The individual observations remain skewed; repeated sample means are the quantities whose distribution changes.

Reference: the retained discrete-sampling comparison

For the lecture’s plants, $X\sim N(21,90)$ and $n=10$, so $\bar X\sim N(21,9)$ exactly and $P(18<\bar X<27)=\Phi(2)-\Phi(-1)\approx0.819$. In contrast, sixty iid $B(9,0.5)$ counts have $\bar X\approx N(4.5,0.0375)$ by CLT; the source’s upper tail .00491 is approximate, while the exact binomial total gives .004301767. The worked examples below retain both methods and their qualifications.

This comparison retains the lecture’s discrete sampling example and its exact-versus-approximate qualification. For normal-probability calculations in the main worked example, identify whether the quantity is an individual observation or a sample mean.