The concept of a joint probability function for two discrete random variables generalises to \(n\) random variables.

Definition

The joint probability function for \(n\) random variables \(\{X_1,X_2,\dots, X_n\}\) is

\[ p(x_1, \dots, x_n) \;=\; P(X_1=x_1 \textbf{ and } \cdots \textbf{ and } X_n=x_n) \]

Maximum likelihood can again be used to estimate any unknown parameters. The likelihood function is the probability of observing the recorded data, treated as a function of the unknown parameter(s). For a single unknown parameter, \(\theta\),

\[ L(\theta \; | \; x_1, x_2, \dots, x_n) \;=\; p(x_1, x_2, \dots, x_n \;| \; \theta) \]

The maximum likelihood estimate of the parameter is the value of the parameter that maximise this.

Independence and random samples

Independence of n random variables

Discrete random variables \(\{X_1,X_2,\dots, X_n\}\) are independent if and only if

\[ p(x_1, \dots, x_n) \;\;=\;\; p_{X_1}(x_1) \times \;\cdots \; \times p_{X_n}(x_n) \qquad \text{ for all } x_1,\dots,x_n \]

In particular, when all \(n\) variables are independent with the same distribution, they are a random sample from this distribution. Their joint probability function was what we maximised earlier when estimating parameters from a random sample by maximum likelihood.