Linear Combinations of Random Variables

A linear combination of random variables is something like $3X + 2$, $X - Y$ or $X_1 + X_2 + X_3$: random quantities scaled, shifted, added and subtracted. Totals, differences, costs and conversions are all linear combinations, so this topic is the engine behind a large share of Paper 6, from "the total mass of five parcels" to the distribution of a sample mean. This note gives the rules for means and variances, which work for any distribution; the follow-up notes deal with the special cases where the distribution of the combination is also known (normal and Poisson).

Scaling and shifting one variable

Suppose $X$ is the temperature in degrees Celsius at noon, with mean $20$ and variance $16$. In Fahrenheit the temperature is $F = 1.8X + 32$.

Shifting by $32$ moves every value up by $32$, so it moves the mean up by $32$. It does not change how spread out the values are, so it does not change the variance.

Scaling by $1.8$ multiplies every value, and every distance from the mean, by $1.8$. So it multiplies the mean by $1.8$ and the standard deviation by $1.8$. Variance is measured in squared units, so it is multiplied by $1.8^2 = 3.24$.

For the temperatures: $E(F) = 1.8 \times 20 + 32 = 68$ and $\text{Var}(F) = 3.24 \times 16 = 51.84$, so the standard deviation is $7.2$, which is $1.8 \times 4$ as expected.

Two consequences are worth spelling out.

• Adding a constant never changes the variance: $\text{Var}(X + 100) = \text{Var}(X)$.
• A negative multiplier still increases or keeps the spread: $\text{Var}(-X) = (-1)^2\,\text{Var}(X) = \text{Var}(X)$, and $\text{Var}(5 - 2X) = 4\,\text{Var}(X)$. Variances can never be negative.

Combining two variables

Now take two random variables $X$ and $Y$, for example the scores of two players.

Means always add. On average, the total of two scores is the total of the two averages, whatever the relationship between the players. So $E(X + Y) = E(X) + E(Y)$ and, more generally, $E(aX + bY) = aE(X) + bE(Y)$.

Variances add when the variables are independent. If the two scores have nothing to do with each other, then sometimes one is high while the other is low and they partly cancel, but just as often both are high or both low. On balance, the total is more variable than either score on its own. For independent variables the variances add exactly.

Differences have added variances

Put $a = 1$ and $b = -1$:

$$E(X - Y) = E(X) - E(Y), \qquad \text{Var}(X - Y) = \text{Var}(X) + (-1)^2\,\text{Var}(Y) = \text{Var}(X) + \text{Var}(Y).$$

The variance of a difference is the sum of the variances. This feels wrong the first time you see it, so think of an example: if the arrival time of your train is uncertain and the departure time of your connection is uncertain, the gap between them is more uncertain than either, not less. Subtracting does not cancel randomness; it adds a second source of it.

Sums of several observations versus a multiple of one

This is the single most important distinction in the topic.

Let $X$ be the mass of one apple. Consider:

• $X_1 + X_2 + X_3 + X_4$: the total mass of four different apples, each with the same distribution as $X$, independently;
• $4X$: four times the mass of one apple.

Both have mean $4E(X)$. But their variances are very different:

$$\text{Var}(X_1 + X_2 + X_3 + X_4) = 4\,\text{Var}(X), \qquad \text{Var}(4X) = 16\,\text{Var}(X).$$

With four different apples, a heavy one is likely to be offset by a lighter one, so the total varies less than four times one apple's mass would. With $4X$ there is no offsetting: one heavy apple is counted four times.

Combining different distributions

The rules hold whatever the distributions are, so you can mix binomial, Poisson and other variables, using their known means and variances:

Distribution	Mean	Variance
$B(n, p)$	$np$	$np(1 - p)$
$\text{Po}(\lambda)$	$\lambda$	$\lambda$
$N(\mu, \sigma^2)$	$\mu$	$\sigma^2$
$\text{Geo}(p)$	$\dfrac{1}{p}$	(not required)

Working backwards and using $E(X^2)$

Questions can give you facts about a combination and ask for the original parameters. Two tools help.

• Set up equations from the expectation and variance rules and solve them simultaneously.
• Remember $\text{Var}(X) = E(X^2) - [E(X)]^2$, so $E(X^2) = \text{Var}(X) + [E(X)]^2$. The rules say nothing directly about $E(X^2)$ or $E(XY)$; you get them through the variance.

Practice questions

Answers

1. $E(4X - 5) = 4(7) - 5 = 23$; $\text{Var}(4X - 5) = 16(3) = 48$; $\text{Var}(5 - 2X) = (-2)^2(3) = 12$.

2. (a) $E(X - Y) = 8$; $\text{Var}(X - Y) = 16 + 9 = 25$, so the standard deviation is $5$. (b) $E(2X + 3Y) = 40 + 36 = 76$; $\text{Var}(2X + 3Y) = 4(16) + 9(9) = 64 + 81 = 145$.

3. $E(X) = 0.3 + 0.4 = 0.7$; $E(X^2) = 0.3 + 0.8 = 1.1$; $\text{Var}(X) = 1.1 - 0.49 = 0.61$. $\text{Var}(X_1 + X_2) = 2(0.61) = 1.22$; $\text{Var}(2X) = 4(0.61) = 2.44$.

4. Total $T = X_1 + \cdots + X_{10}$: $E(T) = 30$, $\text{Var}(T) = 10 \times 3 = 30$. $10X$ would be ten times one day's letters, with variance $100 \times 3 = 300$. The total over ten different days is a sum of separate independent counts, so high and low days partly balance out, giving the smaller variance.

5. $E(X) = 12$, $\text{Var}(X) = 30(0.4)(0.6) = 7.2$; $E(Y) = \text{Var}(Y) = 5$. $E(3X - 2Y + 4) = 36 - 10 + 4 = 30$; $\text{Var}(3X - 2Y + 4) = 9(7.2) + 4(5) = 64.8 + 20 = 84.8$.

6. $E(F) = 1.8(20) + 32 = 68$; standard deviation $= 1.8 \times 4 = 7.2$.

7. $\text{Var}(Y) = a^2(2.25) = 1 \Rightarrow a = \dfrac{1}{1.5} = \dfrac{2}{3}$ (positive root). $E(Y) = \tfrac{2}{3}(6) + b = 0 \Rightarrow b = -4$. (This is standardising: $Y = \dfrac{X - 6}{1.5}$.)

8. (a) $E(F) = 3 + 1.5(8) + 0.4(5) = 3 + 12 + 2 = 17$. $\text{Var}(F) = 1.5^2(9) + 0.4^2(16) = 20.25 + 2.56 = 22.81$; standard deviation $= \sqrt{22.81} = 4.78$ dollars. (b) Total of $50$ separate fares: mean $= 50 \times 17 = 850$; variance $= 50 \times 22.81 = 1140.5$; standard deviation $= 33.8$ dollars. (c) Mean fare $= \dfrac{\text{total}}{50}$, so its variance is $\dfrac{1140.5}{50^2} = 0.456$ (equivalently $\dfrac{22.81}{50}$).