Normal approximation
Earlier confidence intervals for parameters were based on point estimates that are approximately normally distributed.
Given an estimate of the standard error of the estimator, an approximate confidence interval can be obtained from the quantiles of the normal distribution. For example, an approximate 95% CI for a parameter \(\theta\) is
\[ \hat{\theta} - 1.96\; \se(\hat{\theta}) \;\;\lt\;\; \theta \;\;\lt\;\; \hat{\theta} + 1.96\; \se(\hat{\theta}) \]Poisson distribution example
The following table describes the number of heart attacks in a city in 10 weeks.
Week | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
---|---|---|---|---|---|---|---|---|---|---|
Count | 6 | 11 | 13 | 10 | 21 | 8 | 16 | 6 | 9 | 19 |
Assuming a constant rate of heart attacks per week, \(\lambda\), this can be modelled as a random sample from a \(\PoissonDistn(\lambda)\) distribution. The MLE of \(\lambda\) is
\[ \hat{\lambda} \;\;=\;\; \overline{x} \;\;=\;\; 11.9 \]Since the variance of the Poisson distribution is \(\lambda\), we can use the Central Limit Theorem to show that \(\hat \lambda\) is approximately normally distributed in large samples and has standard error
\[ \se(\hat{\lambda}) \;\;=\;\; \sqrt{\frac{\hat{\lambda}}{n}} \;\;=\;\; 1.091 \]This justifies using a normal approximation to find an approximate 95% confidence interval,
\[ \hat{\lambda} \pm 1.96\; \se(\hat{\lambda}) \;\;=\;\; 11.9 \pm 1.96 \times 1.091 \;\;=\;\; 9.76 \text{ to } 14.04 \]Maximum likelihood estimators
This can be used for all maximum likelihood estimators. In large samples, a parameter's MLE is approximately normally distributed with standard error,
\[ \se(\hat {\theta}) \;\;\approx\;\; \sqrt {- \frac 1 {\ell''(\hat {\theta})}} \]This leads to 95% confidence intervals of the form
\[ \hat{\theta} \pm 1.96 \sqrt {- \frac 1 {\ell''(\hat {\theta})}} \]The constant 1.96 can be replaced by other quantiles from the normal distribution to give other confidence levels. Confidence intervals that are found in this way are called Wald-type confidence intervals.