Sum of independent random variables

The result on the previous page gives us the mean and variance for the sum of two independent random variables, \(X_1\) and \(X_2\) (setting \(a = b = 1\)).

\[\begin{aligned} E[X_1 + X_2] \;\;& =\;\; E[X_1] + E[X_2] \\[0.5em] \Var(X_1 + X_2) \;\;& =\;\; \Var(X_1) + \Var(X_2) \end{aligned} \]

If \(X_1\) and \(X_2\) also have the same distributions (and are hence independent identically distributed random variables — i.i.d.r.v.s) with mean \(\mu\) and variance \(\sigma^2\), then this simplifies even further:

\[\begin{aligned} E[X_1 + X_2] \;\;& =\;\; 2\mu \\ \Var(X_1 + X_2) \;\;& =\;\; 2\sigma^2 \end{aligned} \]

Random sample

The result can be extended to give formulae for the mean and variance of the sum of \(n\) i.i.d.r.vs — the sum of the values in a random sample.

Sum of values in a random sample

If \(\{X_1, X_2, ..., X_n\}\) is a random sample of \(n\) values from a discrete distribution with mean \(\mu\) and variance \(\sigma^2\), then the sum of the values, \(\sum_{i=1}^n {X_i}\) has mean and variance

\[\begin{aligned} E\left[\sum_{i=1}^n {X_i}\right] & = n\mu \\ \Var\left(\sum_{i=1}^n {X_i}\right) & = n\sigma^2 \end{aligned} \]

If we write \(\sum_{i=1}^n {X_i} = \sum_{i=1}^{n-1} {X_i} + X_n\), then these two terms are independent. The result at the top of this page shows that

\[\begin{aligned} E\left[\sum_{i=1}^n {X_i}\right] & = E\left[\sum_{i=1}^{n-1} {X_i}\right] + E[X_n] = E\left[\sum_{i=1}^{n-1} {X_i}\right] + \mu \\ \Var\left(\sum_{i=1}^n {X_i}\right) & = \Var\left(\sum_{i=1}^{n-1} {X_i}\right) + \Var(X_n) = \Var\left(\sum_{i=1}^{n-1} {X_i}\right) + \sigma^2 \end{aligned} \]

We have already shown that the result holds when \(n=2\), and these formulae show that if it holds for a sample of \((n-1)\) values, then it also holds for a random sample of size \(n\), completing a proof by induction.

Since the sample mean is simply the sum of the values divided by the constant \(n\), this result also provides us with formulae for the mean and variance of the sample mean.

Mean of a random sample

If \(\{X_1, X_2, ..., X_n\}\) is a random sample of \(n\) values from a discrete distribution with mean \(\mu\) and variance \(\sigma^2\), then the sample mean has a distribution with mean and variance

\[\begin{aligned} E[\overline{X}] \;\;& =\;\; \mu \\ \Var(\overline{X}) \;\;& =\;\; \frac {\sigma^2} n \end{aligned} \]

We showed earlier that for any random variable \(X\),

\[ E[a + b X] = a + b \times E[X] \spaced{and} \Var(a + b \times X) = b^2 \times \Var(X) \]

Applying this to the mean and variance of the sum of values in the random sample with \(a = 0\) and \(b = \diagfrac 1 n\) proves the result.