We end this section by showing how a likelihood ratio test can be inverted to find a confidence interval.

Likelihood ratio test for a single parameter, \(\theta\)

Consider a likelihood ratio test for the hypotheses

A likelihood ratio test is based on the log-likelihood when the null hypothesis holds, \(\ell(\theta_0)\), and its maximum possible value under the alternative hypothesis, \(\ell(\hat{\theta})\), where \(\hat{\theta}\) is the maximum likelihood estimate of \(\theta\).

For a hypothesis test with 5% significance level, we reject the null hypothesis if

\[ 2\left(\ell(\hat{\theta}) - \ell(\theta_0)\right) \;\;\gt\;\; K \]

where the constant \(K\) is the 95th percentile of the \(\ChiSqrDistn(1 \text{ df})\) distribution. From Excel, we can find that \(K = 3.841\).

Inverting the likelihood ratio test

A 95% confidence interval for \(\theta\) can therefore be found as the values for which

\[ \begin{align} 2\left(\ell(\hat{\theta}) - \ell(\theta_0)\right) \;\;&\le\;\; 3.841 \\[0.4em] \ell(\theta_0)\;\;&\gt\;\; \ell(\hat{\theta}) - \frac{3.841}{2} \\[0.5em] \ell(\theta_0)\;\;&\gt\;\; \ell(\hat{\theta}) - 1.921 \end{align} \]

The confidence interval is therefore the values of \(\theta\) for which the log-likelihood is within 1.921 of its maximum.

Example

Clinical records give the survival time in months from diagnosis of 30 sufferers from a certain disease as

9.73
5.56
4.28
4.87
1.55
6.20
1.08
7.17
28.65
6.10
16.16
9.92
2.40
6.19
7.67
1.11
4.66
4.35
7.31
3.28
13.38
3.08
0.41
4.33
2.16
4.49
0.75
4.45
10.29
0.90

If the survival times are exponentially distributed with a death rate of \(\lambda\), find a 95% confidence interval for \(\lambda\).

The log-likelihood for exponentially distributed data is

\[ \ell(\lambda) \;\;=\;\; \sum_{i=1}^n \log f(x_i\;|\;\lambda) \;\;=\;\; n\log(\lambda) - \lambda \sum {x_i} \]

and the maximum likelihood estimate of \(\lambda\) is

\[ \hat{\lambda} \;\;=\;\; \frac n{\sum {x_i}} \;\;=\;\; \frac {30} {182.48} \;\;=\;\; 0.1644\]

The maximum possible value for the log-likelihood is therefore

\[ \ell(\hat{\lambda}) \;\;=\;\; 30\log(0.1644) - 0.1644 \times 182.48 \;\;=\;\; -84.16 \]

A 95% confidence interval for \(\lambda\) therefore consists of values \(\lambda_0\) such that

\[ \ell(\lambda_0)\;\;\gt\;\; \ell(\hat{\lambda}) - 1.921 \;\;=\;\; -86.084 \\[0.4em] 30\times \log \lambda_0 - 182.48 \times \lambda_0 \;\;\gt\;\; -86.084 \]

By trial-and-error, this can be evaluated to be

\[ 0.1124 \;\;\lt\;\; \lambda \;\;\lt\;\; 0.2304 \]

Illustration of the procedure

The diagram below illustrates the procedure.

Firstly consider testing the null hypothesis that \(\lambda = 0.200\). The lower half of the diagram shows the corresponding \(\ExponDistn(\lambda=0.2)\) distribution's pdf. The corresponding log-likelihood is 0.616 below the maximum possible for an exponential model, resulting in a p-value of 0.2671, so we would accept that \(\lambda = 0.200\) at the 5% significance level. The parameter value \(\lambda = 0.200\) is therefore also in a 95% confidence interval for \(\lambda\).

Use the slider to perform the likelihood ratio test for other values of \(\lambda\). Observe that the p-value is greater than 0.05 when

\[ 0.1124 \;\;\lt\;\; \lambda \;\;\lt\;\; 0.2304 \]

so this is a 95% confidence interval for \(\lambda\).

Observe that this also corresponds to parameter values for which the log-likelihood is within 1.921 of its maximum possible value.