Properties of maximum likelihood estimator

From the properties of a sample mean,

\[ E[\hat{\lambda}] \;=\; E[X] \;=\; \lambda \]

so the estimator is unbiased. Its standard error is

\[ \se(\hat{\lambda}) \;=\; \sqrt {\Var(\overline{X})} \;=\; \sqrt {\frac {\Var(X)} n} \;=\; \sqrt{\frac {\lambda} n} \]

From the Central Limit Theorem (or from the asymptotic properties of MLEs), we also know that \(\hat{\lambda}\) is approximately normal in large samples.

Normal-based confidence interval

An approximate 95% confidence interval for any parameter is the estimate ± 1.96 standard errors. Applying this to the maximum likelihood estimate of \(\lambda\), and replacing \(\lambda\) by its estimate, \(\hat{\lambda} = \overline{x}\) in the formula for the standard error, gives the following 95% CI:

\[ \hat{\lambda} \pm 1.96 \times \se(\hat{\lambda})\;\; = \; \; \overline{x} \pm 1.96 \sqrt{\frac {\overline{x}} n} \]

Example: Deaths by horse kicks

Applying this to the Prussian data on the previous page, there were \(\sum {x_i} = 196\) deaths in total out of the \(n = 280\) corps-years, giving a maximum likelihood estimate of

\[ \hat{\lambda} \;=\; \frac {\sum{x_i}} n \;=\; 0.7 \]

Our best estimate of the rate of horse-kick deaths is therefore 0.7 deaths per corps per year.

A 95% confidence interval is

\[ \hat{\lambda} \pm 1.96 \times \se(\hat{\lambda}) \;\; =\;\; 0.7 \pm 1.96 \sqrt{\frac {0.7} {280}} \;\; =\;\; 0.7 \pm 0.098 \]

We are therefore 95% confident that the underlying rate of horse-kick deaths, \(\lambda\), was between 0.602 and 0.798 deaths per corps per year.