Models with two unknown parameters
Most of the flexible models that we described earlier involved two unknown parameters.
\[ X \;\;\sim\;\; \NegBinDistn(\kappa, \pi) \\ X \;\;\sim\;\; \BetaBinomDistn(\alpha, \beta) \\ X \;\;\sim\;\; \WeibullDistn(\alpha, \lambda) \\ X \;\;\sim\;\; \GammaDistn(\alpha, \beta) \\ X \;\;\sim\;\; \NormalDistn(\mu, \sigma^2) \]Method of moments for two parameters
We introduced the method of moments as a way to estimate a single unknown parameter, \(\theta\). The method of moments estimate was the value that made the distribution's mean equal to the mean of a random sample — in other words, by solving
\[ \mu(\theta) \;\;=\;\; \overline{x} \]This can be easily extended to models with two parameters if the parameters are chosen to make the distribution's mean and variance both equal to those from a random sample.
Definition
If a distribution has two unknown parameters, \(\theta\) and \(\phi\), the method of moments estimates of the parameters are found by solving
\[ \mu(\theta, \phi) \;=\; \overline{x} \spaced{and} \sigma^2(\theta, \phi) \;=\; s^2 \]where \(\mu(\theta, \phi)\) and \(\sigma^2(\theta, \phi)\) are the mean and variance of the distribution and \(\overline{x}\) and \(s^2\) are the mean and variance of a random sample from it.
We now illustrate the method with two examples. The first is almost trivially simple.
Example: Normal distribution
If \(X\) has a normal distribution,
\[ X \;\;\sim\;\; \NormalDistn(\mu, \sigma^2) \]then the method of moments estimates of its parameters are
\[ \hat{\mu} = \overline{x} \spaced{and} \hat{\sigma}^2 = s^2 \]The second example is a little harder.
Example: Negative binomial distribution
If \(X\) has a generalised negative binomial distribution,
\[ X \;\;\sim\;\; \NegBinDistn(\kappa, \pi) \]what are the method of moments estimates of \(\kappa\) and \(\pi\) from a random sample?
The distribution's mean and variance are
\[ E[X] = \mu = \frac {\kappa(1-\pi)} \pi \spaced{and} \Var(X) = \sigma^2 = \frac {\kappa(1-\pi)} {\pi^2} \]The method of moments estimates of \(\kappa\) and \(\pi\) are therefore found by solving
\[ \begin{align} \frac {\kappa(1-\pi)} \pi \;&=\; \overline{x} \\ \frac {\kappa(1-\pi)} {\pi^2} \;&=\; s^2 \end{align} \]Dividing the first of these equations by the second gives
\[ \hat{\pi} \;\;=\;\; \frac{\overline{x}}{s^2} \]Substituting back into the first equation gives
\[ \hat{\kappa} \;\;=\;\; \frac{\overline{x}^2}{s^2 - \overline{x}} \]Three or more parameters
Although the method of moments works fairly intuitively for models with one or two parameters, it does not extend easily to models with three or more parameters.