If the value of one variable is known, it may provide information about the likely values of the other variable. This is captured by the conditional probabilities about \(Y\) given \(X\).

\[ P(Y = y \mid X=x) \;\;=\;\; \frac{P(Y = y \text{ and } X=x)}{P(X=x)} \;\;=\;\; \frac{p(x,y)}{p_X(x)} \]

Definition

The conditional distribution of \(Y\) given \(X=x\) is the distribution with probability function

\[ p_{Y \mid X=x}(y) \;\;=\;\; \frac{p(x,y)}{p_X(x)} \]

Note that there are separate conditional distributions of \(Y\) for each possible value of \(X\).

The conditional distribution of \(X\) given \(Y=y\) can be similarly defined as

\[p_{X \mid Y=y}(x) = \large\frac{p(x,y)}{p_Y(y)}\]

Minimum and maximum of three dice

When three fair six-sided dice are rolled, the joint probability function of the minimum, \(Y\), and maximum, \(X\), the joint probabilities are shown in tabular form below.

  Maximum, x
Minimum, y 1 2 3 4 5 6
1 \(\small\diagfrac 1{6^3}\) \(\small\diagfrac 1{6^2}\) \(\small\diagfrac 2{6^2}\) \(\small\diagfrac 3{6^2}\) \(\small\diagfrac 4{6^2}\) \(\small\diagfrac 5{6^2}\)
2 0 \(\small\diagfrac 1{6^3}\) \(\small\diagfrac 1{6^2}\) \(\small\diagfrac 2{6^2}\) \(\small\diagfrac 3{6^2}\) \(\small\diagfrac 4{6^2}\)
3 0 0 \(\small\diagfrac 1{6^3}\) \(\small\diagfrac 1{6^2}\) \(\small\diagfrac 2{6^2}\) \(\small\diagfrac 3{6^2}\)
4 0 0 0 \(\small\diagfrac 1{6^3}\) \(\small\diagfrac 1{6^2}\) \(\small\diagfrac 2{6^2}\)
5 0 0 0 0 \(\small\diagfrac 1{6^3}\) \(\small\diagfrac 1{6^2}\)
6 0 0 0 0 0 \(\small\diagfrac 1{6^3}\)

We will now find the conditional distribution of the maximum value, \(Y\), if it is known that the minimum is \(X = 3\). The marginal probability for \(Y = 3\) is the sum of the probabilities in the highlighted row.

\[ p_Y(3) = \sum_{x=1}^{6} p(x,y) = \frac 1{6^3} + \frac 1 6 \]

The conditional probabilities for \(X\) divide the highlighted row by this value,

\[ p_{X\mid Y=3}(x) =\frac {p(x,3)}{p_X(3)} \]

Because of how \(p_{X\mid Y=3}(x)\) was calculated, the row of conditional probabilities adds to one, making it a valid univariate probability function for \(X\).

Conditional distribution of X, given y = 3
1 2 3 4 5 6
0 0 \(\displaystyle \frac {\small\diagfrac 1{6^3}}{\frac 1{6^3} + \frac 1 6}\) \(\displaystyle \frac {\small\diagfrac 1{6^2}} {\frac 1{6^3} + \frac 1 6}\) \(\displaystyle \frac {\small\diagfrac 2{6^2}} {\frac 1{6^3} + \frac 1 6}\) \(\displaystyle \frac {\small\diagfrac 3{6^2}} {\frac 1{6^3} + \frac 1 6}\)

Conditional mean and variance

Definition

The conditional mean of \(Y\) given \(X=x\) is

\[ E[Y \mid X=x] \;\;=\;\; \sum_{\text{all }y} {y \times p_{Y \mid X=x}(y)} \;\;=\;\; \sum_{\text{all }y} {y \times \frac{p(x,y)}{p_X(x)}} \]

The conditional variance of \(Y\) given \(X=x\) is similarly defined as the variance of this conditional distribution.

Both can depend on the x-value that we are conditioning on.