Joint probability function of \(n\) variables

The concept of a joint probability function for two discrete random variables generalises in an obvious way to one for \(n\) random variables.

Definition

The joint probability function for \(n\) random variables \(\{X_1,X_2,\dots, X_n\}\) is

\[ p(x_1, \dots, x_n) \;=\; P(X_1=x_1 \textbf{ and } \cdots \textbf{ and } X_n=x_n) \]

Such joint distributions often involve unknown parameters and maximum likelihood can again be used to find parameter estimates. The likelihood function is the probability of observing the recorded data, treated as a function of the unknown parameter(s). For a single unknown parameter, \(\theta\), this is

\[ L(\theta \; | \; x_1, x_2, \dots, x_n) \;=\; p(x_1, x_2, \dots, x_n \;| \; \theta) \]

The maximum likelihood estimate of the parameter is the value of the parameter that maximise this.

Independence and random samples

Independence again arises when the joint probability function factorises into a product of the marginal probability functions of the separate variables.

Independence of n random variables

Discrete random variables \(\{X_1,X_2,\dots, X_n\}\) are independent if and only if

\[ p(x_1, \dots, x_n) \;\;=\;\; p_{X_1}(x_1) \times \;\cdots \; \times p_{X_n}(x_n) \qquad \text{ for all } x_1,\dots,x_n \]

In particular, when all \(n\) variables are independent with the same distribution, they are a random sample from this distribution. Their joint probability function was what we maximised earlier when estimating parameters from a random sample by maximum likelihood.