Expected values involving two random variables have similar properties to those of functions of a single random variable. In particular,
Properties of expected values
For any functions of two random variables, \(g(X,Y)\) and \(h(X,Y)\), and constants \(a\) and \(b\),
We will only prove the first of these results, and only for discrete random variables.
\[ \begin{align} E\big[a + b\times g(X,Y)\big] \;&=\; \sum_{\text{all }x} {\sum_{\text{all }y}{\big(a+b\times g(X,Y)\big) \times p(x,y)}} \\ &=\; a \times \sum_{\text{all }x} {\sum_{\text{all }y}{p(x,y)}} \;+\; b\times \sum_{\text{all }x} {\sum_{\text{all }y}{g(x,y)p(x,y)}}\\ &=\; a + b\times E[g(X,Y)] \end{align} \]since \(p(x,y)\) must sum over all \(x\) and \(y\) to one.
The corresponding proof for continuous random variables is the same, but with double integrals replacing the double summations. The second result is proved in a similar way.
Conditional expected values
Conditional expected values are defined in a similar way to unconditional ones, but are based on univariate conditional probability or probability density functions.
Definition
If \(X\) and \(Y\) are discrete random variables, the conditional expected value of \(g(X,Y)\), given that \(X = x\) is
\[ E[g(X,Y) \mid X = x] \;\;=\;\; \sum_{\text{all }y} g(x,y) p_{Y \mid X=x}(y) \]where \(p_{Y \mid X=x}(y)\) is the conditional probability function of \(Y\) given \(X=x\).
If \(X\) and \(Y\) are continuous random variables, the definition is similar with the conditional probability density function replacing \(p_{Y \mid X=x}(y)\) and integration replacing summation.
Note here that \(E[g(X,Y) \mid X = x]\) is a function of \(x\). In a similar way, \(E[g(X,Y) \mid Y = y]\) is a function of \(y\).
Expected values from conditional expected values
Conditional expected values are sometimes useful as an intermediate step to finding unconditional expected values, using the following result.
Unconditional expected values from conditional ones
For any functions of two random variables, \(g(X,Y)\),
\[ E\big[g(X,Y)\big] \;\;=\;\; E \Big[E\big[g(X,Y) \mid X\big] \Big] \]where the outer expectation is over the marginal distribution of \(X\) and the inner expectation is over the conditional distribution of \(Y\) given \(X\).
We only prove the result for discrete random variables. The continuous proof is the same but with integrals instead of summations. The proof starts from the definition of bivariate expected values, noting that the joint probability function can be written as the product of the marginal probability function of \(X\) and the conditional probability function of \(Y\) given \(X\).
\[ \begin{align} E[g(X,Y)] \;&=\; \sum_{\text{all }x} {\sum_{\text{all }y}{g(X,Y) \times p(x,y)}} \\ &=\; \sum_{\text{all }x} {\sum_{\text{all }y}{g(X,Y) \times p_X(x) \times p_{Y \mid X=x}(y)}} \\ &=\; \sum_{\text{all }x} {p_X(x) \left[ \sum_{\text{all }y}{g(X,Y) \times p_{Y \mid X=x}(y)}\right]} \\ &=\; \sum_{\text{all }x} {p_X(x) \times E\big[g(x,Y)\big]} \\ &=\; E \Big[E\big[g(X,Y) \mid X\big] \Big] \end{align} \]