Unbiased Estimates of Mean and Variance

A2 · S2 · 14 min

The population mean $\mu$ and variance $\sigma^2$ are almost never known. What you have is a sample, and from it you must produce the best single-number guesses for $\mu$ and $\sigma^2$ . The sample mean does the job for $\mu$ . For $\sigma^2$ there is a catch: the variance you learnt in S1, which divides by $n$ , comes out too small on average, so Paper 6 uses a version that divides by $n - 1$ . Almost every confidence interval and test question starts with this calculation, often from summarised or coded totals, and it is worth several easy marks if done carefully.

Estimators and estimates

A parameter such as $\mu$ is a fixed but unknown number describing the population. To estimate it you use a formula applied to sample data. Before the sample is taken, that formula gives a random variable called an estimator; once the data are in, it gives a number called an estimate.

For example, the estimator of $\mu$ is the sample mean $\bar{X}$ , a random variable. If your sample of loaves has mean $803.2$ g, then $803.2$ is the estimate.

Because an estimator is a random variable, it has a distribution. Some samples give estimates above the true value, some below. The question is whether the method is right on average.

Definition

An estimator is unbiased if its expected value equals the population parameter it estimates. Although individual estimates vary from sample to sample, the process gives the correct value on average.

An estimate calculated using an unbiased estimator is called an unbiased estimate.

The syllabus only asks for this simple understanding. You will not be asked to prove that an estimator is unbiased.

Estimating the mean

You know from the distribution of the sample mean that $E(\bar{X}) = \mu$ . So the sample mean is an unbiased estimator of the population mean, and

\bar{x} = \frac{\sum x}{n}

is the unbiased estimate of $\mu$ . Nothing new is needed.

Estimating the variance: why $n - 1$

The natural guess for $\sigma^2$ is the variance of the sample, calculated as in S1:

\frac{\sum (x - \bar{x})^2}{n}.

This is biased. It measures spread around $\bar{x}$ , the centre of the sample, and $\bar{x}$ is always exactly in the middle of the sample values. The true mean $\mu$ is usually somewhere else, and the sum of squared deviations is smallest when measured from $\bar{x}$ . So deviations from $\bar{x}$ are, on average, a little too small, and the estimate of the variance is too small.

A small example shows the size of the effect. A spinner gives $1$ , $2$ or $3$ with equal probability, so its variance is $\sigma^2 = \tfrac{2}{3}$ . Take samples of size $n = 2$ . For a sample $(a, b)$ the variance dividing by $n$ is $\left(\dfrac{a - b}{2}\right)^2$ , and dividing by $n - 1$ it is $\dfrac{(a - b)^2}{2}$ . Over the nine equally likely samples:

Difference $\lvert a - b\rvert$	$0$	$1$	$2$
Number of samples	$3$	$4$	$2$
Variance, dividing by $n$	$0$	$0.25$	$1$
Variance, dividing by $n - 1$	$0$	$0.5$	$2$

The averages are

\frac{3(0) + 4(0.25) + 2(1)}{9} = \frac{1}{3} \qquad\text{and}\qquad \frac{3(0) + 4(0.5) + 2(2)}{9} = \frac{2}{3}.

Dividing by $n$ gives $\tfrac{1}{3}$ on average, which is too small. Dividing by $n - 1$ gives $\tfrac{2}{3}$ on average: exactly $\sigma^2$ . In general, dividing by $n$ gives on average $\dfrac{n - 1}{n}\sigma^2$ , and multiplying by $\dfrac{n}{n - 1}$ removes the bias.

Unbiased estimates

From a random sample $x_1, x_2, \dots, x_n$ :

\bar{x} = \frac{\sum x}{n}

s^2 = \frac{1}{n - 1}\sum (x - \bar{x})^2 = \frac{1}{n - 1}\left(\sum x^2 - \frac{\left(\sum x\right)^2}{n}\right)

$\bar{x}$ is the unbiased estimate of $\mu$ and $s^2$ is the unbiased estimate of $\sigma^2$ . Both formulae are in the formula booklet.

If the variance of the sample (dividing by $n$ ) is already known, then

s^2 = \frac{n}{n - 1} \times (\text{variance of the sample}).

For large samples the factor $\dfrac{n}{n - 1}$ is close to $1$ and the two versions are nearly equal. For small samples the difference is significant, and in every case the mark scheme expects $n - 1$ .

The forms the data come in

Questions give the data in one of four ways. The method is always the same: get $n$ , $\sum x$ and $\sum x^2$ (or their coded versions), then substitute.

Raw data. Add up the values and their squares, or use the statistics mode of your calculator. Most calculators show two standard deviations: one labelled $\sigma_n$ or $\sigma_x$ (dividing by $n$ ) and one labelled $s_{n-1}$ or $s_x$ (dividing by $n - 1$ ). Use the second, and square it.

Summary totals. You are given $n$ , $\sum x$ and $\sum x^2$ . Substitute directly.

Coded totals. You are given $\sum (x - a)$ and $\sum (x - a)^2$ for some constant $a$ . Subtracting a constant shifts every value by the same amount, which moves the mean but leaves the spread unchanged. So

\bar{x} = a + \frac{\sum (x - a)}{n}, \qquad s^2 = \frac{1}{n - 1}\left(\sum (x - a)^2 - \frac{\left(\sum (x - a)\right)^2}{n}\right).

A frequency table. Use $\sum fx$ and $\sum fx^2$ in place of $\sum x$ and $\sum x^2$ , with $n = \sum f$ .

Unbiased estimates of mean and variance

Write down $n$ , $\sum x$ and $\sum x^2$ (or $\sum (x - a)$ and $\sum (x - a)^2$ , or $\sum fx$ and $\sum fx^2$ ).
Mean: $\bar{x} = \dfrac{\sum x}{n}$ , adding back $a$ if the data are coded.
Variance: $s^2 = \dfrac{1}{n - 1}\left(\sum x^2 - \dfrac{(\sum x)^2}{n}\right)$ , with coded totals in place of $\sum x$ and $\sum x^2$ if given. Never add $a$ to the variance.
Give answers to 3 significant figures (or exactly), and keep more figures if the value is used later.

Raw data

A random sample of $6$ packets of rice has masses, in grams,

498,\quad 503,\quad 501,\quad 497,\quad 505,\quad 502.

Find unbiased estimates of the population mean and variance.

Solution

$n = 6$ , $\sum x = 3006$ , $\sum x^2 = 1\,506\,052$ .

\bar{x} = \frac{3006}{6} = 501

s^2 = \frac{1}{5}\left(1\,506\,052 - \frac{3006^2}{6}\right) = \frac{1}{5}(1\,506\,052 - 1\,506\,006) = \frac{46}{5} = 9.2

Check with deviations from $501$ : $-3, 2, 0, -4, 4, 1$ , whose squares add to $9 + 4 + 0 + 16 + 16 + 1 = 46$ . Dividing by $5$ gives $9.2$ . With large numbers like these, the deviation form avoids subtracting two huge, nearly equal totals.

Summary totals and coded totals

(a) For a random sample of $50$ values of $x$ , $\sum x = 1240$ and $\sum x^2 = 31\,900$ . Find unbiased estimates of the population mean and variance.

(b) The times, $t$ seconds, taken by a random sample of $40$ students to solve a puzzle are summarised by $\sum (t - 30) = 64$ and $\sum (t - 30)^2 = 1130$ . Find unbiased estimates of the population mean and variance of $t$ .

Solution

(a)

\bar{x} = \frac{1240}{50} = 24.8, \qquad s^2 = \frac{1}{49}\left(31\,900 - \frac{1240^2}{50}\right) = \frac{1}{49}(31\,900 - 30\,752) = \frac{1148}{49} = 23.4

(b) The mean of the coded values is $\dfrac{64}{40} = 1.6$ , so

\bar{t} = 30 + 1.6 = 31.6.

The variance is unaffected by subtracting $30$ :

s^2 = \frac{1}{39}\left(1130 - \frac{64^2}{40}\right) = \frac{1}{39}(1130 - 102.4) = \frac{1027.6}{39} = 26.3

A frequency table

The numbers of goals scored in a random sample of $50$ football matches are shown.

Goals, $x$	$0$	$1$	$2$	$3$	$4$	$5$
Frequency, $f$	$8$	$15$	$12$	$9$	$4$	$2$

(a) Find unbiased estimates of the mean and variance of the number of goals per match.

(b) Comment on whether a Poisson distribution could be a suitable model.

Solution

(a)

\sum fx = 0 + 15 + 24 + 27 + 16 + 10 = 92, \qquad \sum fx^2 = 0 + 15 + 48 + 81 + 64 + 50 = 258

\bar{x} = \frac{92}{50} = 1.84, \qquad s^2 = \frac{1}{49}\left(258 - \frac{92^2}{50}\right) = \frac{1}{49}(258 - 169.28) = \frac{88.72}{49} = 1.81

(b) For a Poisson distribution the mean equals the variance. The estimates $1.84$ and $1.81$ are close, so a Poisson model is plausible (see modelling with the Poisson distribution).

From a calculator's standard deviation

A calculator gives the standard deviation of a sample of $20$ values, calculated by dividing by $n$ , as $4.5$ . Find the unbiased estimate of the population variance.

Solution

The variance of the sample is $4.5^2 = 20.25$ , so

s^2 = \frac{20}{19} \times 20.25 = 21.3 \text{ (3 s.f.)}

Exam-hard: combining two samples

A random sample of $20$ observations of a random variable $X$ has mean $14.2$ , and the unbiased estimate of the population variance from this sample is $3.6$ . A second, independent random sample of $30$ observations of $X$ gives $\sum x = 441$ and $\sum x^2 = 6600$ .

(a) Using both samples together, find unbiased estimates of the mean and variance of $X$ .

(b) Use your estimates to find the approximate probability that the mean of a further random sample of $40$ observations of $X$ is greater than $15$ .

Solution

(a) Recover the totals for the first sample.

\sum x = 20 \times 14.2 = 284

From $s^2 = \dfrac{1}{n - 1}\left(\sum x^2 - \dfrac{(\sum x)^2}{n}\right)$ :

3.6 = \frac{1}{19}\left(\sum x^2 - \frac{284^2}{20}\right) \;\Rightarrow\; \sum x^2 = 19 \times 3.6 + 4032.8 = 4101.2

Combined: $n = 50$ , $\sum x = 284 + 441 = 725$ , $\sum x^2 = 4101.2 + 6600 = 10\,701.2$ .

\bar{x} = \frac{725}{50} = 14.5

s^2 = \frac{1}{49}\left(10\,701.2 - \frac{725^2}{50}\right) = \frac{1}{49}(10\,701.2 - 10\,512.5) = \frac{188.7}{49} = 3.851

(b) The distribution of $X$ is unknown, but $n = 40$ is large, so by the Central Limit Theorem

\bar{X} \sim N\left(14.5, \frac{3.851}{40}\right) \text{ approximately}, \qquad \text{standard deviation } 0.31028.

P(\bar{X} > 15) = P\left(Z > \frac{0.5}{0.31028}\right) = P(Z > 1.611) = 1 - 0.9464 = 0.0536

You cannot simply average the two variance estimates: the samples have different sizes and different means, and the spread of the combined data includes the gap between those means.

Where the estimates are used

When $\sigma^2$ is unknown and the sample is large, $s^2$ stands in for $\sigma^2$ in confidence intervals and in tests for a population mean. An error in $s^2$ therefore carries through the rest of a long question, which is why the method mark for this step is so valuable.

In the same spirit, the sample proportion $p_s = \dfrac{x}{n}$ is an unbiased estimate of a population proportion $p$ , because $E\left(\dfrac{X}{n}\right) = \dfrac{np}{n} = p$ when $X \sim B(n, p)$ . It is used in confidence intervals for proportions.

Common mistakes

Dividing by $n$ . The unbiased estimate of variance divides by $n - 1$ . Using $n$ loses the accuracy mark, and everything built on it is wrong.
Using the wrong calculator value. $\sigma_n$ (or $\sigma_x$ ) divides by $n$ ; $s_{n-1}$ (or $s_x$ ) divides by $n - 1$ . Square the right one.
Adding the coding constant to the variance. $\sum (x - a)$ needs $a$ added back to get the mean; the variance needs nothing added.
Mixing coded and uncoded totals. In $\sum (x - a)^2 - \dfrac{(\sum (x - a))^2}{n}$ both totals must be coded.
Forgetting to square $\sum x$ . It is $\dfrac{(\sum x)^2}{n}$ , not $\dfrac{\sum x^2}{n}$ .
Giving $s$ when asked for the variance, or $s^2$ when a standard deviation is needed in the next part. Read which one is asked for.
Explaining "unbiased" as "the sample is random" or "the estimate is accurate". It means the expected value of the estimator equals the parameter, so the method is right on average.

Exam tip

The question words "unbiased estimates of the population mean and variance" signal exactly this calculation. Present it as two separate lines, formula then numbers.
Show the substitution into the formula, such as $\dfrac{1}{49}\left(31\,900 - \dfrac{1240^2}{50}\right)$ . A slip after a correct substitution keeps the method mark.
Give answers exactly where they terminate ( $9.2$ , $31.6$ ) or to 3 significant figures. If the value is used later, carry more figures.
When a later part needs the standard deviation, take $s = \sqrt{s^2}$ from the unrounded value.
"Explain what is meant by an unbiased estimate" is worth one mark: "the expected value of the estimator equals the population parameter, so on average it gives the true value".

Summary

An estimator is unbiased if its expected value equals the parameter: right on average, though individual estimates vary.
$\bar{x} = \dfrac{\sum x}{n}$ is the unbiased estimate of $\mu$ .
$s^2 = \dfrac{1}{n - 1}\left(\sum x^2 - \dfrac{(\sum x)^2}{n}\right)$ is the unbiased estimate of $\sigma^2$ .
Dividing by $n$ underestimates $\sigma^2$ on average, by the factor $\dfrac{n - 1}{n}$ .
$s^2 = \dfrac{n}{n - 1} \times$ (variance of the sample, dividing by $n$ ).
For coded data, add $a$ back to the mean only; the variance is unchanged by coding.
For frequency tables use $\sum fx$ , $\sum fx^2$ and $n = \sum f$ .

Practice questions

Question

A random sample gives the values $12, 15, 11, 18, 14$ . Find unbiased estimates of the population mean and variance.
For a random sample of $80$ values, $\sum x = 2000$ and $\sum x^2 = 52\,600$ . Find unbiased estimates of the population mean and variance.
For a random sample of $25$ values, $\sum (x - 100) = -30$ and $\sum (x - 100)^2 = 486$ . Find unbiased estimates of the population mean and variance.
The variance of a random sample of $10$ values, calculated by dividing by $n$ , is $6.3$ . Find the unbiased estimate of the population variance.
The numbers of children in a random sample of $70$ households are shown.

Children, $x$ $0$ $1$ $2$ $3$ $4$
Frequency $12$ $20$ $25$ $10$ $3$

Find unbiased estimates of the population mean and variance.
A researcher says that the sample mean is an unbiased estimator of the population mean. Explain what this means.
A random sample of $12$ observations has mean $5.5$ , and the unbiased estimate of the population variance is $2.4$ . A thirteenth observation, $8$ , is added to the sample. Find the new unbiased estimates of the population mean and variance.
For a random sample of $40$ values of $x$ , $\sum (x - k) = 48$ and $\sum (x - k)^2 = 900$ , where $k$ is a constant. (a) Find the unbiased estimate of the population variance. (b) The unbiased estimate of the population mean is $21.2$ . Find $k$ .

Children, $x$	$0$	$1$	$2$	$3$	$4$
Frequency	$12$	$20$	$25$	$10$	$3$

Answers

$\sum x = 70$ , $\sum x^2 = 1010$ , $n = 5$ . $\bar{x} = 14$ ; $s^2 = \tfrac{1}{4}\left(1010 - \tfrac{70^2}{5}\right) = \tfrac{1}{4}(1010 - 980) = 7.5$ .
$\bar{x} = \tfrac{2000}{80} = 25$ ; $s^2 = \tfrac{1}{79}\left(52\,600 - \tfrac{2000^2}{80}\right) = \tfrac{1}{79}(52\,600 - 50\,000) = \tfrac{2600}{79} = 32.9$ .
$\bar{x} = 100 + \tfrac{-30}{25} = 98.8$ ; $s^2 = \tfrac{1}{24}\left(486 - \tfrac{(-30)^2}{25}\right) = \tfrac{1}{24}(486 - 36) = 18.75$ .
$s^2 = \tfrac{10}{9} \times 6.3 = 7$ .
$\sum f = 70$ , $\sum fx = 0 + 20 + 50 + 30 + 12 = 112$ , $\sum fx^2 = 0 + 20 + 100 + 90 + 48 = 258$ . $\bar{x} = \tfrac{112}{70} = 1.6$ ; $s^2 = \tfrac{1}{69}\left(258 - \tfrac{112^2}{70}\right) = \tfrac{1}{69}(258 - 179.2) = \tfrac{78.8}{69} = 1.14$ .
The expected value of the sample mean equals the population mean, $E(\bar{X}) = \mu$ . Different samples give different sample means, some too high and some too low, but on average the sample mean equals the population mean.
Original: $\sum x = 12 \times 5.5 = 66$ and $\sum x^2 = 11 \times 2.4 + \tfrac{66^2}{12} = 26.4 + 363 = 389.4$ . New: $n = 13$ , $\sum x = 74$ , $\sum x^2 = 389.4 + 64 = 453.4$ . $\bar{x} = \tfrac{74}{13} = 5.69$ ; $s^2 = \tfrac{1}{12}\left(453.4 - \tfrac{74^2}{13}\right) = \tfrac{1}{12}(453.4 - 421.23) = 2.68$ .
(a) $s^2 = \tfrac{1}{39}\left(900 - \tfrac{48^2}{40}\right) = \tfrac{1}{39}(900 - 57.6) = \tfrac{842.4}{39} = 21.6$ . Note that $k$ is not needed: coding does not affect the variance. (b) $\bar{x} = k + \tfrac{48}{40} = k + 1.2 = 21.2$ , so $k = 20$ .

Estimators and estimates

Estimating the mean

Estimating the variance: why n−1n - 1n−1

The forms the data come in

Where the estimates are used

Practice questions

Console

Estimating the variance: why $n - 1$