We end this section by showing how a likelihood ratio test can be inverted to find a confidence interval.
Likelihood ratio test for a single parameter, \(\theta\)
Consider a likelihood ratio test for the hypotheses
A likelihood ratio test is based on the log-likelihood when the null hypothesis holds, \(\ell(\theta_0)\), and its maximum possible value under the alternative hypothesis, \(\ell(\hat{\theta})\), where \(\hat{\theta}\) is the maximum likelihood estimate of \(\theta\).
For a hypothesis test with 5% significance level, we reject the null hypothesis if
\[ 2\left(\ell(\hat{\theta}) - \ell(\theta_0)\right) \;\;\gt\;\; K \]where the constant \(K\) is the 95th percentile of the \(\ChiSqrDistn(1 \text{ df})\) distribution. From Excel, we can find that \(K = 3.841\).
Inverting the likelihood ratio test
A 95% confidence interval for \(\theta\) can therefore be found as the values for which
\[ \begin{align} 2\left(\ell(\hat{\theta}) - \ell(\theta_0)\right) \;\;&\le\;\; 3.841 \\[0.4em] \ell(\theta_0)\;\;&\gt\;\; \ell(\hat{\theta}) - \frac{3.841}{2} \\[0.5em] \ell(\theta_0)\;\;&\gt\;\; \ell(\hat{\theta}) - 1.921 \end{align} \]The confidence interval is therefore the values of \(\theta\) for which the log-likelihood is within 1.921 of its maximum.
Example
Clinical records give the survival time in months from diagnosis of 30 sufferers from a certain disease as
9.73 5.56 4.28 4.87 |
1.55 6.20 1.08 7.17 |
28.65 6.10 16.16 9.92 |
2.40 6.19 7.67 1.11 |
4.66 4.35 7.31 3.28 |
13.38 3.08 0.41 4.33 |
2.16 4.49 0.75 |
4.45 10.29 0.90 |
If the survival times are exponentially distributed with a death rate of \(\lambda\), find a 95% confidence interval for \(\lambda\).
The log-likelihood for exponentially distributed data is
\[ \ell(\lambda) \;\;=\;\; \sum_{i=1}^n \log f(x_i\;|\;\lambda) \;\;=\;\; n\log(\lambda) - \lambda \sum {x_i} \]and the maximum likelihood estimate of \(\lambda\) is
\[ \hat{\lambda} \;\;=\;\; \frac n{\sum {x_i}} \;\;=\;\; \frac {30} {182.48} \;\;=\;\; 0.1644\]The maximum possible value for the log-likelihood is therefore
\[ \ell(\hat{\lambda}) \;\;=\;\; 30\log(0.1644) - 0.1644 \times 182.48 \;\;=\;\; -84.16 \]A 95% confidence interval for \(\lambda\) therefore consists of values \(\lambda_0\) such that
\[ \ell(\lambda_0)\;\;\gt\;\; \ell(\hat{\lambda}) - 1.921 \;\;=\;\; -86.084 \\[0.4em] 30\times \log \lambda_0 - 182.48 \times \lambda_0 \;\;\gt\;\; -86.084 \]By trial-and-error, this can be evaluated to be
\[ 0.1124 \;\;\lt\;\; \lambda \;\;\lt\;\; 0.2304 \]Illustration of the procedure
The diagram below illustrates the procedure.
Firstly consider testing the null hypothesis that \(\lambda = 0.200\). The lower half of the diagram shows the corresponding \(\ExponDistn(\lambda=0.2)\) distribution's pdf. The corresponding log-likelihood is 0.616 below the maximum possible for an exponential model, resulting in a p-value of 0.2671, so we would accept that \(\lambda = 0.200\) at the 5% significance level. The parameter value \(\lambda = 0.200\) is therefore also in a 95% confidence interval for \(\lambda\).
Use the slider to perform the likelihood ratio test for other values of \(\lambda\). Observe that the p-value is greater than 0.05 when
\[ 0.1124 \;\;\lt\;\; \lambda \;\;\lt\;\; 0.2304 \]so this is a 95% confidence interval for \(\lambda\).
Observe that this also corresponds to parameter values for which the log-likelihood is within 1.921 of its maximum possible value.