Hypothesis Tests for a Binomial Proportion

A2 · S2 · 16 min

Is a coin biased? Has a new treatment raised the recovery rate? Do fewer customers now use a discount code? Each of these is a claim about a probability $p$ , and each can be tested with a single observation: the number of successes in a sample of size $n$ . That number has a binomial distribution, so the test uses exact binomial probabilities. Paper 6 asks you to carry out such tests, to find critical regions and actual significance levels, and to handle two-tailed tests, usually as a prelude to finding the probabilities of Type I and Type II errors.

The set-up

A sample of $n$ independent trials is observed, each with the same probability $p$ of success. The test statistic is the number of successes, $X$ . If the null hypothesis $H_0: p = p_0$ is true, then

X \sim B(n, p_0).

The alternative hypothesis is $p > p_0$ , $p < p_0$ or $p \ne p_0$ , depending on the wording of the question (see the nature of hypothesis testing). Large values of $X$ are evidence for $p > p_0$ ; small values are evidence for $p < p_0$ .

There are no binomial tables in the 9709 formula booklet. You calculate probabilities from

P(X = r) = \binom{n}{r}p^r(1 - p)^{n - r}

or with your calculator's binomial functions. Questions are usually designed so that the tail you need has only a few terms, but either way you must show the expression you are evaluating.

Method 1: compare a probability with the significance level

Calculate the probability, under $H_0$ , of the observed value or anything more extreme in the direction of $H_1$ .

For $H_1: p > p_0$ with observed value $x$ , find $P(X \ge x)$ .
For $H_1: p < p_0$ with observed value $x$ , find $P(X \le x)$ .
For $H_1: p \ne p_0$ , find the tail on the side of the observed value (compare $x$ with the mean $np_0$ to decide which), and compare with half the significance level.

If the probability is less than the significance level, reject $H_0$ .

The reason for using $P(X \ge x)$ rather than $P(X = x)$ is that any single value can be unlikely. With $n = 100$ , even the most likely outcome has a small probability. What matters is whether the observation is far out in the tail, and a tail probability measures exactly that.

Binomial test using a tail probability

Define $p$ in context and write $H_0: p = p_0$ and $H_1$ .
State "under $H_0$ , $X \sim B(n, p_0)$ ", defining $X$ .
Write down the tail probability for the observed value, in the direction of $H_1$ , as a sum of terms or as $1 - P(X \le x - 1)$ .
Compare with the significance level (half of it for a two-tailed test), showing the inequality.
State whether $H_0$ is rejected, then conclude in context without certainty.

Routine: a lower-tail test

In the general population $10\%$ of people are left-handed. A researcher suspects that left-handedness is less common among basketball players. In a random sample of $50$ basketball players, $2$ are left-handed. Test the researcher's suspicion at the $5\%$ significance level.

Solution

Let $p$ be the proportion of basketball players who are left-handed.

$H_0: p = 0.1$ , $\quad H_1: p < 0.1$ .

Let $X$ be the number of left-handed players in the sample. Under $H_0$ , $X \sim B(50, 0.1)$ .

P(X \le 2) = 0.9^{50} + \binom{50}{1}(0.1)(0.9)^{49} + \binom{50}{2}(0.1)^2(0.9)^{48}

= 0.00515 + 0.02863 + 0.07794 = 0.1117

$0.1117 > 0.05$ , so do not reject $H_0$ .

There is insufficient evidence at the $5\%$ level that left-handedness is less common among basketball players.

An upper-tail test

A standard treatment for a condition is successful for $60\%$ of patients. A new treatment is tried on a random sample of $20$ patients and is successful for $17$ of them. Test at the $5\%$ significance level whether the new treatment has a higher success rate.

Solution

Let $p$ be the probability that the new treatment is successful for a patient.

$H_0: p = 0.6$ , $\quad H_1: p > 0.6$ .

Let $X$ be the number of successes. Under $H_0$ , $X \sim B(20, 0.6)$ .

P(X \ge 17) = \binom{20}{17}(0.6)^{17}(0.4)^3 + \binom{20}{18}(0.6)^{18}(0.4)^2 + \binom{20}{19}(0.6)^{19}(0.4) + (0.6)^{20}

= 0.01235 + 0.00309 + 0.00049 + 0.00004 = 0.0160

$0.0160 < 0.05$ , so reject $H_0$ .

There is evidence at the $5\%$ level that the new treatment has a higher success rate than the standard treatment.

Method 2: find the critical region

The critical region is the set of values of $X$ that would lead to rejecting $H_0$ . Because $X$ is discrete, you cannot usually hit the significance level exactly. The rule is:

Critical regions for a discrete test statistic

Lower tail, $H_1: p < p_0$ : the critical region is $X \le c$ , where $c$ is the largest value with $P(X \le c) \le \alpha$ .
Upper tail, $H_1: p > p_0$ : the critical region is $X \ge c$ , where $c$ is the smallest value with $P(X \ge c) \le \alpha$ .
Two-tailed, $H_1: p \ne p_0$ : find a lower region and an upper region, each with probability as close as possible to, but not more than, $\tfrac{1}{2}\alpha$ .

The actual significance level is the probability of the critical region when $H_0$ is true. It is at most $\alpha$ , and usually less.

To justify a critical region you must show the cumulative probability for the boundary value and for the next value along, which is too big. That pair of numbers is what proves $c$ is right.

The graph shows $B(20, 0.3)$ as bars of width $1$ , with the lower-tail critical region $X \le 2$ at the $5\%$ level shaded. Its total probability is $0.0355$ ; including $X = 3$ would take it to $0.1071$ , which is too much.

y = fact(20) / (fact(floor(x + 0.5)) * fact(20 - floor(x + 0.5))) * 0.3^floor(x + 0.5) * 0.7^(20 - floor(x + 0.5)) + 0*sqrt(x + 0.5) + 0*sqrt(20.5 - x) fill -0.5 2.5 y = fact(20) / (fact(floor(x + 0.5)) * fact(20 - floor(x + 0.5))) * 0.3^floor(x + 0.5) * 0.7^(20 - floor(x + 0.5))

Finding a critical region

State $H_0$ , $H_1$ and the distribution under $H_0$ .
Build up the cumulative probabilities from the end of the relevant tail: $P(X = 0)$ , $P(X \le 1)$ , $P(X \le 2)$ , ... (or $P(X = n)$ , $P(X \ge n - 1)$ , ... for the upper tail).
Stop when the cumulative probability first exceeds the significance level (or half of it, for each tail of a two-tailed test).
State the critical region, quoting the last probability that was within the limit and the first that was not.
If asked, the actual significance level is the probability of the critical region.

A lower-tail critical region

It is claimed that $30\%$ of customers at a shop use a discount code. The manager suspects that the proportion is lower. She will test the claim at the $5\%$ significance level using a random sample of $20$ customers.

(a) Find the critical region for the test.

(b) State the actual significance level.

Solution

(a) Let $p$ be the proportion of customers who use a code. $H_0: p = 0.3$ , $\quad H_1: p < 0.3$ . Under $H_0$ , $X \sim B(20, 0.3)$ .

P(X = 0) = 0.7^{20} = 0.00080

P(X \le 1) = 0.00080 + 20(0.3)(0.7)^{19} = 0.00080 + 0.00684 = 0.00764

P(X \le 2) = 0.00764 + 190(0.3)^2(0.7)^{18} = 0.00764 + 0.02785 = 0.0355

P(X \le 3) = 0.0355 + 1140(0.3)^3(0.7)^{17} = 0.0355 + 0.0716 = 0.1071

$P(X \le 2) = 0.0355 \le 0.05$ but $P(X \le 3) = 0.1071 > 0.05$ , so the critical region is $X \le 2$ .

(b) The actual significance level is $P(X \le 2) = 0.0355$ , that is $3.55\%$ .

(c) $X = 3$ is not in the critical region, so do not reject $H_0$ . There is insufficient evidence at the $5\%$ level that fewer than $30\%$ of customers use a discount code.

A two-tailed critical region

A survey says $30\%$ of households in a town own a cat. A vet wants to test whether this proportion has changed, using a random sample of $20$ households and a $5\%$ significance level.

(a) Find the critical region.

(b) Find the actual significance level of the test.

Solution

(a) $H_0: p = 0.3$ , $\quad H_1: p \ne 0.3$ . Under $H_0$ , $X \sim B(20, 0.3)$ . Each tail may have probability at most $0.025$ .

Lower tail:

P(X \le 1) = 0.0076 \le 0.025, \qquad P(X \le 2) = 0.0355 > 0.025.

So the lower part of the critical region is $X \le 1$ .

Upper tail:

P(X \ge 11) = 0.0171 \le 0.025, \qquad P(X \ge 10) = 0.0480 > 0.025.

So the upper part is $X \ge 11$ .

The critical region is $X \le 1$ or $X \ge 11$ .

(b) The actual significance level is

P(X \le 1) + P(X \ge 11) = 0.0076 + 0.0171 = 0.0248.

The two tails need not be equal, and the actual level is below $5\%$ because neither tail can reach exactly $2.5\%$ .

Exam-hard: when can a test reject at all?

A company claims that $40\%$ of customers prefer its brand. A rival believes the proportion is lower and plans a test at the $5\%$ significance level, based on the number $X$ of customers in a random sample of $n$ who prefer the brand.

(a) Show that if $n = 5$ , the test can never lead to rejecting the company's claim.

(b) Find the smallest value of $n$ for which it is possible to reject the claim.

(d) In fact, $4$ of the $20$ customers prefer the brand. State the conclusion of the test.

Solution

(a) $H_0: p = 0.4$ , $\quad H_1: p < 0.4$ . The most extreme result possible is $X = 0$ . With $n = 5$ ,

P(X = 0) = 0.6^5 = 0.0778 > 0.05,

so even the most extreme result is not in the critical region, and $H_0$ can never be rejected.

(b) Rejection is possible only if $P(X = 0) = 0.6^n \le 0.05$ .

0.6^n \le 0.05 \;\Rightarrow\; n \ge \frac{\ln 0.05}{\ln 0.6} = 5.86

Check: $0.6^5 = 0.0778$ and $0.6^6 = 0.0467$ . The smallest value is $n = 6$ .

P(X \le 3) = 0.0160 \le 0.05, \qquad P(X \le 4) = 0.0510 > 0.05.

The critical region is $X \le 3$ , and the actual significance level is $0.0160$ , that is $1.60\%$ .

Notice how far below $5\%$ this is: $P(X \le 4)$ is only just over $0.05$ , but it is over, so $X = 4$ cannot be included.

(d) $X = 4$ is not in the critical region. Do not reject $H_0$ : there is insufficient evidence at the $5\%$ level that fewer than $40\%$ of customers prefer the brand.

Common mistakes

Using $P(X = x)$ instead of a tail probability. For "increased", find $P(X \ge x)$ ; for "decreased", $P(X \le x)$ .
Off-by-one errors in the complement. $P(X \ge 17) = 1 - P(X \le 16)$ , not $1 - P(X \le 17)$ .
Choosing the wrong tail in a two-tailed test. Compare the observed value with the mean $np_0$ . Above the mean, use the upper tail; below it, the lower tail.
Not halving the level for a two-tailed test. Each tail of a $5\%$ two-tailed test has $2.5\%$ .
Picking the critical value whose probability is closest to $5\%$ . The critical region's probability must not exceed the significance level, even if a larger region would be closer.
Justifying a critical region with one probability. Show the boundary value's cumulative probability ( $\le \alpha$ ) and the next one ( $> \alpha$ ).
Saying the actual significance level is $5\%$ . For a discrete test it is the probability of the critical region, such as $3.55\%$ .

Exam tip

Define $p$ in context ("the proportion of customers who use a code"). Hypotheses in terms of $X$ or the sample are not accepted.
"Under $H_0$ , $X \sim B(20, 0.3)$ " earns a mark. Write it.
Write tail probabilities as expressions before evaluating. If you use a calculator's cumulative function, still state which probability you found, such as $P(X \le 2) = 0.0355$ .
For "find the critical region", the answer is a set of values of $X$ , written as an inequality such as $X \le 2$ or $X \ge 11$ . Do not give a probability as the answer.
The comparison and conclusion follow the same rules as every test: show the inequality, state the decision about $H_0$ , and conclude in context with "evidence" language.
Many questions continue by asking for $P(\text{Type I error})$ , which is the actual significance level you have just found.

Summary

Test statistic: the number of successes $X$ ; under $H_0$ , $X \sim B(n, p_0)$ .
Compare $P(X \ge x)$ or $P(X \le x)$ (in the direction of $H_1$ ) with the significance level, halved for each tail of a two-tailed test.
Lower-tail critical region: largest $c$ with $P(X \le c) \le \alpha$ . Upper-tail: smallest $c$ with $P(X \ge c) \le \alpha$ .
Show the cumulative probabilities either side of the boundary.
The actual significance level is the probability of the critical region under $H_0$ , at most $\alpha$ .
If even the most extreme outcome has probability above $\alpha$ , the test can never reject $H_0$ .
Conclude in context with non-definite language.

Practice questions

Question

Find the critical region for a test of $H_0: p = 0.4$ against $H_1: p > 0.4$ at the $5\%$ significance level, using $X \sim B(15, p)$ . State the actual significance level.
A blood group is found in $25\%$ of the population. In a random sample of $30$ people from a particular region, $3$ have this blood group. Test at the $5\%$ significance level whether the blood group is less common in this region.
A spinner is designed so that the probability of red is $\tfrac{1}{3}$ . In $18$ spins, red occurs $10$ times. Test at the $5\%$ significance level whether the spinner is biased towards red.
A coin is to be tested for bias using $12$ tosses and a two-tailed test at the $5\%$ level. Find the critical region and the actual significance level.
A test of $H_0: p = 0.3$ against $H_1: p < 0.3$ at the $1\%$ significance level is based on a random sample of size $n$ . Find the smallest value of $n$ for which $H_0$ could be rejected.
It is thought that $20\%$ of emails received by a company are spam. After a filter is updated, a manager believes the proportion of spam getting through has increased. A random sample of $20$ emails is checked, and a test is carried out at the $10\%$ significance level. (a) Find the critical region. (b) The sample contains $6$ spam emails. State the conclusion of the test.
A politician claims that at least $55\%$ of voters support her. An opponent believes that support is lower. In a random sample of $16$ voters, $5$ support the politician. Test the opponent's belief at the $2.5\%$ significance level.
A test of $H_0: p = 0.35$ against $H_1: p \ne 0.35$ uses a random sample of size $20$ , and $H_0$ is rejected if $X \le 2$ or $X \ge 12$ . (a) Find the significance level of the test. (b) Show that this is the critical region for a two-tailed test at the $5\%$ significance level. (c) In the sample, $X = 12$ . State the conclusion.

Answers

Under $H_0$ , $X \sim B(15, 0.4)$ . $P(X \ge 10) = 0.0338 \le 0.05$ and $P(X \ge 9) = 0.0950 > 0.05$ . Critical region $X \ge 10$ ; actual significance level $0.0338$ .
$p$ = proportion in the region with the blood group. $H_0: p = 0.25$ , $H_1: p < 0.25$ . Under $H_0$ , $X \sim B(30, 0.25)$ . $P(X \le 3) = 0.75^{30} + 30(0.25)(0.75)^{29} + 435(0.25)^2(0.75)^{28} + 4060(0.25)^3(0.75)^{27}$ $= 0.00018 + 0.00179 + 0.00863 + 0.02685 = 0.0374$ . $0.0374 < 0.05$ : reject $H_0$ . There is evidence at the $5\%$ level that the blood group is less common in this region.
$p$ = probability of red. $H_0: p = \tfrac{1}{3}$ , $H_1: p > \tfrac{1}{3}$ . Under $H_0$ , $X \sim B\left(18, \tfrac{1}{3}\right)$ . $P(X \ge 10) = 1 - P(X \le 9) = 0.0433$ . $0.0433 < 0.05$ : reject $H_0$ . There is evidence at the $5\%$ level that the spinner is biased towards red.
$H_0: p = 0.5$ , $H_1: p \ne 0.5$ , $X \sim B(12, 0.5)$ , each tail at most $0.025$ . $P(X \le 2) = \tfrac{1 + 12 + 66}{4096} = 0.0193 \le 0.025$ ; $P(X \le 3) = \tfrac{299}{4096} = 0.0730 > 0.025$ . Lower region $X \le 2$ ; by symmetry the upper region is $X \ge 10$ . Critical region $X \le 2$ or $X \ge 10$ ; actual significance level $2 \times 0.0193 = 0.0386$ .
Need $P(X = 0) = 0.7^n \le 0.01$ , so $n \ge \dfrac{\ln 0.01}{\ln 0.7} = 12.91$ . Check: $0.7^{12} = 0.0138$ , $0.7^{13} = 0.00969$ . Smallest $n = 13$ .
(a) $H_0: p = 0.2$ , $H_1: p > 0.2$ , $X \sim B(20, 0.2)$ . $P(X \ge 7) = 0.0867 \le 0.1$ ; $P(X \ge 6) = 0.1958 > 0.1$ . Critical region $X \ge 7$ . (b) $6$ is not in the critical region. Do not reject $H_0$ : insufficient evidence at the $10\%$ level that the proportion of spam getting through has increased.
$p$ = proportion of voters who support her. $H_0: p = 0.55$ , $H_1: p < 0.55$ . Under $H_0$ , $X \sim B(16, 0.55)$ . $P(X \le 5) = 0.0486$ (sum of the six terms $r = 0$ to $5$ ). $0.0486 > 0.025$ : do not reject $H_0$ . There is insufficient evidence at the $2.5\%$ level that her support is lower than $55\%$ .
(a) Under $H_0$ , $X \sim B(20, 0.35)$ . $P(X \le 2) = 0.0121$ and $P(X \ge 12) = 0.0196$ . Significance level $= 0.0121 + 0.0196 = 0.0317$ . (b) Lower: $P(X \le 2) = 0.0121 \le 0.025$ and $P(X \le 3) = 0.0444 > 0.025$ . Upper: $P(X \ge 12) = 0.0196 \le 0.025$ and $P(X \ge 11) = 0.0532 > 0.025$ . So these are the largest regions in each tail with probability at most $2.5\%$ . (c) $X = 12$ is in the critical region. Reject $H_0$ : there is evidence at the $5\%$ level that the proportion is not $0.35$ . (The result is in the upper tail, which suggests that if the proportion has changed, it has increased.)

The set-up

Method 1: compare a probability with the significance level

Method 2: find the critical region

Practice questions

Console