Our earlier definition of independence of two random variables, \(X\) and \(Y\), was when all events about \(X\) are independent of all events about \(Y\). For continuous random variables, this is equivalent to the joint probability density function factorising into the marginal pdf of \(x\) times the marginal pdf of \(Y\).
Independence
Two continuous random variables, X, and Y, are independent if and only if
\[ f(x, y) = f_X(x) \times f_Y(y) \qquad \text{ for all } x \text{ and } y \]This result was described and justified less formally in an earlier page.
If \(X\) and \(Y\) are independent, then the conditional pdf of \(X\), given that \(Y = y\) is equal to its marginal pdf.
\[ f_{X\mid Y=y}(x) \;\;=\;\; \frac {f(x,y)}{f_Y(y)} \;\;=\;\; f_X(x) \]This means that all conditional distributions of \(X\) are the same — they are not affected by the known value \(y\) — so knowing the value of \(Y\) gives no information about the value of \(X\). Similarly,
\[ f_{Y\mid X=x}(y) \;\;=\;\; f_Y(y) \]so knowing the value of \(X\) gives no information about \(Y\).
Determining independence
Independence is an important concept. Sometimes we can deduce mathematically that two variables are independent by factorising their joint pdf.
However independence is more often justified by the context from which the two variables were defined.
Failure of light bulbs
If two light bulbs are tested at 80ºC until failure, their failure times \(X\) and \(Y\) can be assumed to be independent — failure of one bulb would not influence when the other failed. If the distribution for a single light bulb is \(\ExponDistn(\lambda)\), there joint pdf would therefore be
\[ f(x, y) = f_X(x) \times f_Y(y) = \left(\lambda e^{\lambda x}\right)\left(\lambda e^{\lambda y}\right) = \lambda^2 e^{\lambda(x+y)}\qquad \text{ if } x \ge 0\text{ and } y \ge 0 \]Extensions to 3 or more variables
The idea of a joint probability density function for three or more continuous random variables \(\{X_1,X_2,\dots, X_n\}\) is a simple extension of that for two variables,
\[ f(x_1, \dots, x_n) \]Probabilities can be obtained from the joint pdf as multiple integrals over the corresponding values of the variables, \((x_1, \dots, x_n)\), but we will not go into the details here.
Random samples
The most important practical application of distributions involving several random variables, arises when we have a collection of \(n\) independent random variables from the same distribution — a random sample from the distribution. The joint pdf of the variables is then
\[ f(x_1, \dots, x_n) \;=\; \prod_{i=1}^n f(x_i) \]where \(f(\cdot)\) is the pdf of the distribution from which the random sample is taken. This joint pdf is what we maximise when estimating parameters by maximum likelihood.