Models with two unknown parameters

Most of the flexible models that we described earlier involved two unknown parameters.

\[ X \;\;\sim\;\; \NegBinDistn(\kappa, \pi) \\ X \;\;\sim\;\; \BetaBinomDistn(\alpha, \beta) \\ X \;\;\sim\;\; \WeibullDistn(\alpha, \lambda) \\ X \;\;\sim\;\; \GammaDistn(\alpha, \beta) \\ X \;\;\sim\;\; \NormalDistn(\mu, \sigma^2) \]

Method of moments for two parameters

We introduced the method of moments as a way to estimate a single unknown parameter, \(\theta\). The method of moments estimate was the value that made the distribution's mean equal to the mean of a random sample — in other words, by solving

\[ \mu(\theta) \;\;=\;\; \overline{x} \]

This can be easily extended to models with two parameters if the parameters are chosen to make the distribution's mean and variance both equal to those from a random sample.

Definition

If a distribution has two unknown parameters, \(\theta\) and \(\phi\), the method of moments estimates of the parameters are found by solving

\[ \mu(\theta, \phi) \;=\; \overline{x} \spaced{and} \sigma^2(\theta, \phi) \;=\; s^2 \]

where \(\mu(\theta, \phi)\) and \(\sigma^2(\theta, \phi)\) are the mean and variance of the distribution and \(\overline{x}\) and \(s^2\) are the mean and variance of a random sample from it.

We now illustrate the method with two examples. The first is almost trivially simple.

Example: Normal distribution

If \(X\) has a normal distribution,

\[ X \;\;\sim\;\; \NormalDistn(\mu, \sigma^2) \]

then the method of moments estimates of its parameters are

\[ \hat{\mu} = \overline{x} \spaced{and} \hat{\sigma}^2 = s^2 \]

The second example is a little harder.

Example: Negative binomial distribution

If \(X\) has a generalised negative binomial distribution,

\[ X \;\;\sim\;\; \NegBinDistn(\kappa, \pi) \]

what are the method of moments estimates of \(\kappa\) and \(\pi\) from a random sample?

The distribution's mean and variance are

\[ E[X] = \mu = \frac {\kappa(1-\pi)} \pi \spaced{and} \Var(X) = \sigma^2 = \frac {\kappa(1-\pi)} {\pi^2} \]

The method of moments estimates of \(\kappa\) and \(\pi\) are therefore found by solving

\[ \begin{align} \frac {\kappa(1-\pi)} \pi \;&=\; \overline{x} \\ \frac {\kappa(1-\pi)} {\pi^2} \;&=\; s^2 \end{align} \]

Dividing the first of these equations by the second gives

\[ \hat{\pi} \;\;=\;\; \frac{\overline{x}}{s^2} \]

Substituting back into the first equation gives

\[ \hat{\kappa} \;\;=\;\; \frac{\overline{x}^2}{s^2 - \overline{x}} \]

Three or more parameters

Although the method of moments works fairly intuitively for models with one or two parameters, it does not extend easily to models with three or more parameters.