← STAT 20070

Section 3 · Lecture 15 · Hypothesis Testing

Hypothesis Testing

A hypothesis test checks a claim about a population using a sample: we assume the claim is true and ask whether data like ours would be surprising. This page sets out the idea and the five steps, works through one example, lets you change the alternative hypothesis, the significance level and the sample mean to see how the decision changes, shows where the p-value comes from by simulation, and ends with a short quiz.

1

What is a hypothesis test?

A way to check a claim about a population (a mean, a proportion) using a sample. We assume the claim is true and ask: would data like ours be surprising?

1State two hypotheses

H₀, the null hypothesis: the claim we start from. It always has an equals sign.

HA, the alternative: what we look for evidence of.

H₀: μ = μ₀

2Compute a test statistic

How far is the sample mean from μ₀, in standard errors? If H₀ is true it follows a t-distribution with n − 1 degrees of freedom.

t = (ȳ − μ₀) / (s/√n)

3Ask how unusual it is

The p-value is the probability, if H₀ were true, of a test statistic at least as extreme as the one observed. Small p means strong evidence against H₀.

p ≤ α ⇒ reject H₀

Which alternative? It depends on the question

μ > μ₀Upper-tailed: is it higher than claimed? (pollution above the limit)
μ < μ₀Lower-tailed: is it lower than claimed? (vitamin C below the label)
μ ≠ μ₀Two-tailed: is it different in either direction? (a drug changes blood pressure)

Two ways to be wrong

H₀ is trueH₀ is false
Reject H₀Type I error (probability α)Correct
Do not reject H₀CorrectType II error

Drug trial: a Type I error approves a drug that does not work; a Type II error misses one that does.

The significance level α (usually 0.10, 0.05 or 0.01) is the Type I error risk we accept.E.g. α = 0.05 in the drug trial: if the drug really does not work, there is a 5% chance the trial still concludes it does. Test 100 useless drugs and about 5 would be approved by chance. It is not the chance that an approved drug is useless.t in rejection region ⇔ p ≤ α ⇒ reject H₀otherwise ⇒ do not reject H₀ (not the same as “H₀ is true”)
2

A worked example: the sugar claim

Tutorial 5, Q5. A soft-drink producer says its cans average 11 g of sugar per 100 mL, and you measured 12 cans. Do they give enough evidence to say the producer’s claim is false?

The producer’s claimμ = 11 g / 100 mLThis is the reference value μ₀. It becomes the null hypothesis.
Your sample of n = 12 cansȳ = 11.433, s = 0.4053
3

The five steps, live

Change the alternative hypothesis, the significance level or the sample mean. The red area is the rejection region (probability α); the green area is the p-value. Reject H₀ when t lands in the red, which is exactly when the green area is no bigger than α.

Alternative hypothesis HA
Significance level α

n = 12 and s = 0.4053 stay fixed, so the test uses a t-distribution with 11 degrees of freedom.

Distribution of T = (Ȳ − 11)/(s/√12) if the producer is right: t with 11 df

rejection region (area α)p-value (area beyond your t)your t

    4

    Why is your t so unusual?

    The p-value is not magic. Imagine the producer is right (μ = 11), buy 12 cans again and again, and compute t each time. Count how often t is at least as extreme as yours.

    t from simulated samples if μ = 11as extreme as yourst density, 11 df

    5

    Quick quiz

    Six questions. Pick an answer to see whether it is right and why.

    Score: 0 / 6