Random variables defined from others
We now consider two independent random variables, \(X\) and \(Y\). Any function of these two variables can be used to define another random variable.
\[Z =g(X, Y)\]This has a distribution whose shape depends on those of \(X\) and \(Y\), but we will only consider its mean and variance here.
We will use the following result later.
Product of independent random variables
If two discrete random variables, \(X\) and \(Y\), are independent,
\[ E[XY] = E[X] \times E[Y] \]The expected value of this function is again the sum of its possible values, multiplied by their probabilities,
\[ \begin {align} E[X Y] & = \sum_{\text{all } x, y} {(x y) \times p_{XY}(x,y)} \\ & = \sum_{\text{all } x} \sum_{\text{all } y} {x \times y\times p_X(x)\times p_Y(y)} \quad\quad \text{since }X \text{ and } Y \text{ are independent} \\ & = \left( \sum_{\text{all } x} {x \times p_X(x)} \right) \times \left( \sum_{\text{all } y} {y \times p_Y(y)} \right) \\ & = E[X] \times E[Y] \end {align} \]More important in practice is a linear combination of \(X\) and \(Y\),
\(Z =aX + bY\) where \(a\) and \(b\) are constants
Linear combination of independent variables
If the means of two independent discrete random variables, \(X\) and \(Y\), are \(\mu_X\) and \(\mu_Y\) and their variances are \(\sigma_X^2\) and \(\sigma_Y^2\), then the linear combination \((aX + bY)\) has mean and variance
\[ \begin {align} E[aX + bY] & = a\mu_X + b\mu_Y \\[0.4em] \Var(aX + bY) & = a^2\sigma_X^2 + b^2\sigma_Y^2 \end {align} \]The mean is
\[ \begin {align} E[aX + bY] & = \sum_{\text{all } x, y} {(ax + by)\times p_{XY}(x,y)} \\ & = \sum_{\text{all } x} \sum_{\text{all } y} {(ax + by)\times p_X(x)\times p_Y(y)} \quad\quad \text{since }X \text{ and } Y \text{ are independent} \\ & = a \times \sum_{\text{all } x} \sum_{\text{all } y} {x \times p_X(x)\times p_Y(y)} + b \times \sum_{\text{all } x} \sum_{\text{all } y} {y \times p_X(x)\times p_Y(y)} \\ & = a \times \left(\sum_{\text{all } x} {x \times p_X(x)} \right)\left(\sum_{\text{all } y} {p_Y(y)} \right) + b \times \left(\sum_{\text{all } x} {p_X(x)} \right)\left(\sum_{\text{all } y} {y \times p_Y(y)} \right) \\ & = a \times \mu_X \times 1 + b \times 1 \times \mu_Y \\ & = a \mu_X + b \mu_Y \end {align} \]The variance is the expected value of the squared difference between the linear combination and its mean,
\[ \begin {align} \Var(aX + bY) & = E\left[ \big( (aX + bY) - (a\mu_X + b\mu_Y) \big)^2 \right] \\ & = E\left[ \big( a(X - \mu_X) + b(Y - \mu_Y) \big)^2 \right] \\ & = E\left[ \big(a(X - \mu_X)\big)^2 + \big(b(Y - \mu_Y)\big)^2 + 2ab(X - \mu_X)(Y - \mu_Y) \right] \\ & = a^2 \Var(X) + b^2 \Var(Y) + 2ab E\left[ (X - \mu_X)(Y - \mu_Y) \right] \end {align} \]Now
\[ \begin {align} E\big[ (X - \mu_X)(Y - \mu_Y) \big] & = E[ XY - \mu_X Y - X\mu_Y + \mu_X\mu_Y] \\ & = E[XY] - \mu_X E[Y] - E[X] \mu_Y + \mu_X\mu_Y \\ & = E[X]E[Y] - E[X]E[Y] - E[X]E[Y] + E[X]E[Y]\\ & = 0 \end {align} \]Since we proved earlier that \(E[XY] = E[X]E[Y]\) for independent variables. Therefore
\[ \Var(aX + bY) = a^2 \Var(X) + b^2 \Var(Y) \]Although the formula for the mean still holds if \(X\) and \(Y\) are not independent, the formula for the variance requires modification to cope with dependent random variables.