Use a p-value

A p-value follows the observation

Start with the null distribution and the observed statistic. Shade results at least as extreme in the direction of the alternative, then calculate that tail probability. Compare the unrounded p-value with the chosen significance level.

A p-value is the null probability of a statistic at least as extreme in the direction(s) of $H_1$. With $Z\sim N(0,1)$:

• Left-sided, observed $z=-2$: $\Phi(-2)=0.0227501319\ldots\approx0.0228$.
• Right-sided, observed $z=2$: the upper-tail probability is the same.
• Two-sided, observed $z=2$: $2P(Z\ge2)=0.0455002639\ldots\approx0.0455$.

The source prints the calculator decimals 0.022750062 and 0.045500124; these differ slightly from the independent CDF values above but round to the same 0.0228 and 0.0455. The source calculator entries are normalcdf(−10^99, −2, 0, 1), normalcdf(2, 10^99, 0, 1) and twice the latter for the two-sided case. The very large finite endpoints stand in for infinite tails. Preserve full precision during the calculation.

Make both methods agree

Reject $H_0$ when $p\le\alpha$; otherwise do not reject it. The equivalent critical-value rules reject below the left boundary, above the right boundary, or outside both boundaries for a two-sided test. State the conclusion in the context of the population mean and the chosen level.

At 5%, compare the same decision in both forms:

• Left-sided, $z=-1.5$: $p\approx0.0668>0.05$ and the statistic lies above $-1.645$: do not reject.
• Right-sided, $z=1.5$: $p\approx0.0668$ and the statistic lies below $1.645$: do not reject.
• Two-sided, $z=2$: $p\approx0.0455$ and the statistic exceeds $1.960$: reject. The source’s right-tail non-rejection table includes equality; use $z<z_{\mathrm{crit}}$ consistently with its main inclusive rejection rule.

Compare unrounded values at the boundary

The source discusses strict versus inclusive boundaries and suggests another sample at equality. Here we use the declared rule $p\le\alpha$ consistently; equality has probability zero under a continuous null model. Do not resample repeatedly just to obtain a preferred decision. If rounding makes a result appear equal, compare the unrounded values.

State what the decision means

Not rejecting $H_0$ does not prove or accept it. A test supplies evidence against $H_0$ rather than assigning a probability that it is true. For the same data and alternative, rejection at a smaller level implies rejection at every larger level; the converse fails. Statistical significance does not measure practical importance.

Explain the evidence before making the decision

For a right-sided test of a population mean, an observed sample gives $p=0.032$. Two students say:

• “There is a 3.2% chance that the null hypothesis is true.”
• “If the null model were true, results at least this extreme in the stated direction would occur with probability 0.032.”

Choose the defensible explanation. Decide what happens at 5% and at 1%, and explain why the data have not changed.

After your attempt: compare the reasoning

The second explanation conditions on the null model. Since $0.032<0.05$, reject at 5%; since $0.032>0.01$, do not reject at 1%. The stricter rule requires stronger evidence. Neither decision gives the probability that the null hypothesis is true. A non-rejection does not establish that the claimed mean is correct.