Normal approximation
The confidence intervals for parameters that we described earlier were all based on point estimates that could be assumed to be approximately normally distributed.
Given an estimate of the standard error of the estimator, an approximate confidence interval can be obtained from the quantiles of the normal distribution. For example, an approximate 95% CI for a parameter \(\theta\) is
\[ \hat{\theta} - 1.96\; \se(\hat{\theta}) \;\;\lt\;\; \theta \;\;\lt\;\; \hat{\theta} + 1.96\; \se(\hat{\theta}) \]Poisson distribution example
The following table describes the number of heart attacks in a city in 10 weeks.
Week | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
---|---|---|---|---|---|---|---|---|---|---|
Count | 6 | 11 | 13 | 10 | 21 | 8 | 16 | 6 | 9 | 19 |
Assuming a constant rate of heart attacks per week, \(\lambda\), this can be modelled as a random sample from a \(\PoissonDistn(\lambda)\) distribution. The maximum likelihood and method of moments estimators of \(\lambda\) are both
\[ \hat{\lambda} \;\;=\;\; \overline{x} \;\;=\;\; 11.9 \]Since the variance of the Poisson distribution is \(\lambda\),
\[ \Var(\hat{\lambda}) \;\;=\;\; \frac{\lambda}{n} \]This provides an estimated standard error
\[ \se(\hat{\lambda}) \;\;=\;\; \sqrt{\frac{\hat{\lambda}}{n}} \;\;=\;\; 1.091 \]The Central Limit Theorem shows that the sample mean will also be approximately normal, at least in large samples, justifying a normal approximation to find an approximate 95% confidence interval,
\[ \hat{\lambda} \pm 1.96\; \se(\hat{\lambda}) \;\;=\;\; 11.9 \pm 1.96 \times 1.091 \;\;=\;\; 9.76 \text{ to } 14.04 \]Wald-type confidence interval
One special case of this method applies to maximum likelihood estimates. General theory shows that in large samples, the maximum likelihood estimator of a parameter is approximately normally distributed with a standard error that can be found from the second derivative of the log-likelihood,
\[ \se(\hat {\theta}) \;\;\approx\;\; \sqrt {- \frac 1 {\ell''(\hat {\theta})}} \]This leads to 95% confidence intervals of the form
\[ \hat{\theta} \pm 1.96 \sqrt {- \frac 1 {\ell''(\hat {\theta})}} \]The constant 1.96 can be replaced by other quantiles from the normal distribution to give other confidence levels. Confidence intervals that are found in this way are called Wald-type confidence intervals.