Hypothesis tests for a mean

Match the statistic to its assumptions

A statistic such as $\bar X$ is random before sampling; its observed value $\bar x$ is a number. Under $H_0$, standardise using $(\bar X-\mu_0)/(\sigma/\sqrt n)$ when the population variance is known. Choose the reference model before calculating:

• For independent observations from a normal population with known variance, the sample mean has an exact normal distribution at any sample size.
• For a sufficiently large independent random sample with finite variance, use a justified CLT approximation. If the population variance is unknown, use its unbiased estimate and state that the resulting normal test is approximate.
• A small sample from an unspecified population does not supply a normal reference model merely because a mean and a variance can be calculated.

Define the statistic before substituting its observed value. For a known population standard deviation, write

$$Z=\frac{\bar X-\mu_0}{\sigma/\sqrt n}.$$

Under the exact normal null model, $Z\sim N(0,1)$. Replacing $\sigma$ by $s$ needs the large-sample qualification above.

Reference: the retained variance-case comparison

The source’s four cases need these qualifications:

Normal, known $\sigma^2$, any sample size

Sampling justification: Exact normal sample mean

Reference distribution: Exact standard normal

Non-normal or unspecified, known $\sigma^2$, large random sample

Sampling justification: CLT, with independent observations and finite variance

Reference distribution: Approximately standard normal

Normal, unknown variance estimated by $S^2$

Sampling justification: The source labels this exactly normal for any $n$

Reference distribution: In fact $(\bar X-\mu_0)/(S/\sqrt n)$ has a $t_{n-1}$ distribution; a normal approximation requires large-sample justification

Non-normal or unspecified, unknown variance, large random sample

Sampling justification: CLT and a suitable variance estimate

Reference distribution: Approximately standard normal

The source uses $n\ge30$ as its classroom large-sample guideline; this is not a guarantee for every population. Small-sample tests with unknown variance are not made exact z-tests by assuming normality. Keep this distinction when selecting the method. See NIST’s known-versus-estimated standard deviation distinction.

Significance and errors

Fix the level and alternative before collecting evidence. Allocate the rejection probability to the tail or tails named by the alternative. Keep the long-run error rate separate from any claim about the probability that this particular hypothesis is true.

A significance level describes the testing rule across repeated samples when the null model is true. At $\alpha=0.05$, an exact continuous test is designed to reject on 5% of those repeated samples. It does not assign a 5% probability to the null hypothesis being true or to this particular conclusion being wrong.

Imagine repeatedly checking a filling machine whose true mean is its claimed 500 ml. Before collecting data, fix the alternative and the rejection region. A result in that region supplies evidence against the claim, while leaving the possibility of an unusual sample from a correctly set machine.

Use $\alpha$ as a probability: 5% means $0.05$. For the same alternative and data, reducing the level shrinks the rejection region, so evidence sufficient at 5% may be insufficient at 1%. The test’s stated significance requires its sampling and distribution assumptions to hold.

Reference: terminology used in the retained source

A Type I error rejects a true $H_0$; a Type II error does not reject a false $H_0$. The four source outcomes are:

Do not reject

$H_0$ true: Correct decision

$H_0$ false: Type II error

Reject

$H_0$ true: Type I error

$H_0$ false: Correct decision

Use $\alpha$ as a probability: 5% means $\alpha=0.05$, not 5. In this exact continuous test,

$$P(\text{reject }H_0\mid H_0\text{ true})=\alpha$$

Reducing $\alpha$ makes the rejection region smaller and reduces Type I error; it need not reduce every kind of error. It can increase Type II error. The source’s unqualified “smaller error” sentence must be read as referring to Type I error.

Critical regions and calculator tails

Fix the rejection region from the alternative and $\alpha$ before observing the sample. At 10%, the three rules are:

• Left-sided: reject for $z\le-1.28155$.
• Right-sided: reject for $z\ge1.28155$.
• Two-sided: reject for $|z|\ge1.64485$. The source’s two-sided inequality contains absolute-value bars, although native extraction loses them.

For a 10% two-sided test the central area is 0.90. The source uses invNorm(0.9, 0, 1, CENTER); its distinct Teacher note also teaches invNorm(0.05, 0, 1, LEFT) for the negative boundary and invNorm(0.05, 0, 1, RIGHT) for the positive boundary. These are equivalent ways to allocate 0.05 to each tail. The signs differ, while the magnitudes agree.

A p-value follows the observation

Start with the null distribution and the observed statistic. Shade results at least as extreme in the direction of the alternative, then calculate that tail probability. Compare the unrounded p-value with the chosen significance level.

A p-value is the null probability of a statistic at least as extreme in the direction(s) of $H_1$. With $Z\sim N(0,1)$:

• Left-sided, observed $z=-2$: $\Phi(-2)=0.0227501319\ldots\approx0.0228$.
• Right-sided, observed $z=2$: the upper-tail probability is the same.
• Two-sided, observed $z=2$: $2P(Z\ge2)=0.0455002639\ldots\approx0.0455$.

The source prints the calculator decimals 0.022750062 and 0.045500124; these differ slightly from the independent CDF values above but round to the same 0.0228 and 0.0455. The source calculator entries are normalcdf(−10^99, −2, 0, 1), normalcdf(2, 10^99, 0, 1) and twice the latter for the two-sided case. The very large finite endpoints stand in for infinite tails. Preserve full precision during the calculation.

Make both methods agree

Reject $H_0$ when $p\le\alpha$; otherwise do not reject it. The equivalent critical-value rules reject below the left boundary, above the right boundary, or outside both boundaries for a two-sided test. State the conclusion in the context of the population mean and the chosen level.

At 5%, compare the same decision in both forms:

• Left-sided, $z=-1.5$: $p\approx0.0668>0.05$ and the statistic lies above $-1.645$: do not reject.
• Right-sided, $z=1.5$: $p\approx0.0668$ and the statistic lies below $1.645$: do not reject.
• Two-sided, $z=2$: $p\approx0.0455$ and the statistic exceeds $1.960$: reject. The source’s right-tail non-rejection table includes equality; use $z<z_{\mathrm{crit}}$ consistently with its main inclusive rejection rule.

Compare unrounded values at the boundary

The source discusses strict versus inclusive boundaries and suggests another sample at equality. Here we use the declared rule $p\le\alpha$ consistently; equality has probability zero under a continuous null model. Do not resample repeatedly just to obtain a preferred decision. If rounding makes a result appear equal, compare the unrounded values.

State what the decision means

Not rejecting $H_0$ does not prove or accept it. A test supplies evidence against $H_0$ rather than assigning a probability that it is true. For the same data and alternative, rejection at a smaller level implies rejection at every larger level; the converse fails. Statistical significance does not measure practical importance.

Explain the evidence before making the decision

For a right-sided test of a population mean, an observed sample gives $p=0.032$. Two students say:

• “There is a 3.2% chance that the null hypothesis is true.”
• “If the null model were true, results at least this extreme in the stated direction would occur with probability 0.032.”

Choose the defensible explanation. Decide what happens at 5% and at 1%, and explain why the data have not changed.

After your attempt: compare the reasoning

The second explanation conditions on the null model. Since $0.032<0.05$, reject at 5%; since $0.032>0.01$, do not reject at 1%. The stricter rule requires stronger evidence. Neither decision gives the probability that the null hypothesis is true. A non-rejection does not establish that the claimed mean is correct.

When a test conclusion is given, find the unknown

Translate the stated decision into an inequality before solving for the unknown. Keep the alternative, significance units and inclusive rejection rule fixed. Check the resulting boundary against the original test, especially when the unknown is an integer sample size.

The source identifies five ingredients: the hypothesised mean $\mu_0$, observed mean $\bar x$, sample size $n$, known population variance $\sigma^2$ or its unbiased estimate $s^2$, and significance level.

When the significance percentage is unknown, compute p. If a is the numerical percentage, solve $p\le a/100$: 5% means a=5. Equivalently, compare p with $\alpha$ expressed as a probability, so 5% means $\alpha=0.05$.

When $\mu_0$, $\bar x$, $n$ or $s^2$ is unknown, critical-value inequalities often make the algebra clearer.

These are method-selection guides, not exclusions of equivalent CDF inversion.

For a large random sample, no individual normality assumption is needed for the CLT approximation; independence, finite variance and adequate approximation remain necessary. If the population is normal with known variance, the exact sample-mean distribution works at any n.

Respect the domain of the unknown

For a positive integer sample size, solve the inequality before choosing an integer. For a real-valued mean, preserve the unrounded boundary: printing an endpoint to 3 s.f. does not restrict all possible means to a 3 s.f. grid.