In practical situations involving multinomial distributions, \(\pi_1\), ..., \(\pi_g\) are unknown and must be estimated from data. We now consider estimating their values from a single multinomial observation,
\[ (X_1,\dots,X_g) \;\;\sim\;\; \MultinomDistn(n, \pi_1, ..., \pi_g) \]Likelihood function
The likelihood function is the probability of observing the data, but treated as a function of the unknown parameters.
\[ L(\pi_1, ..., \pi_g \mid x_1,\dots,x_g) \;\;=\;\; \frac{n!}{x_1!\;x_2!\; \cdots,\;x_g!} \pi_1^{x_1}\pi_2^{x_2}\cdots \pi_g^{x_g} \]We can eliminate one of the unknown parameters here since
\[ \pi_g \;\;=\;\; 1 - \pi_1 - \pi_2 - \cdots - \pi_{g-1} \]We will therefore rewrite the likelihood as
\[ L(\pi_1, ..., \pi_{g-1}) \;\;=\;\; \frac{n!}{x_1!\;x_2!\; \cdots,\;x_g!} \pi_1^{x_1}\pi_2^{x_2}\cdots \pi_{g-1}^{x_{g-1}} (1 - \pi_1 - \pi_2 - \cdots - \pi_{g-1})^{x_g} \]The log-likelihood is
\[ \begin{align} \ell(\pi_1, ..., \pi_{g-1}) \;\;=\;\; x_1 \log(\pi_1) + \cdots &+ x_{g-1} \log(\pi_{g-1})\\[0.4em] &+ x_g \log(1 - \pi_1 - \pi_2 - \cdots - \pi_{g-1}) + K \end{align} \]where \(K\) does not depend on the unknown parameters. We can find the maximum likelihood estimates from this.
Maximum likelihood estimates
If \((x_1, x_2, \dots, x_g)\) are a random sample from a \(\MultinomDistn(n, \pi_1, \dots, \pi_g)\) distribution, the maximum likelihood estimates of \(\pi_1, \dots, \pi_g\) are
\[ \hat{\pi}_i \;\;=\;\; \frac{x_i}{n} \](Proved in full version)
These are simply the corresponding sample proportions.