Assumption of normality
The main problem with Wald-type confidence intervals is that they require an assumption that the estimator, \(\hat{\theta}\), has an approximately normal distribution. Although the asymptotic properties of maximum likelihood estimators show that this holds for large sample sizes, the assumption may be violated when the sample size is small.
Pivot
We now describe a better way to get confidence intervals that does not require an assumption of normality. The method is based on a random quantity called a pivot.
Definition
If a random sample, \(\{X_1, X_2, \dots, X_n\}\) is selected from a distribution with unknown parameter \(\theta\), a pivot is a function of the data and \(\theta\) whose distribution is fully known (and therefore does not involve unknown parameters).
\[ g(\theta, X_1, \dots, X_n) \;\;\sim\;\; \mathcal{Distn} \]Since \(\theta\) is unknown, we cannot actually evaluate the function \(g(\theta, X_1, \dots, X_n)\). However its known distribution can be used as the basis of a confidence interval.
Since the distribution \(\mathcal{Distn}\) is fully known, we can evaluate its quantiles. For a 95% confidence interval, we would want the quantiles corresponding to tail probabilities of 2½% and a central probability of 95%. In general, for a \((1 - \alpha)\) confidence interval, we would want the quantiles to cut off a probability \(\dfrac{\alpha}2\) in each tail of the distribution.
For example, if the pivot had a \(\NormalDistn(0, 1)\) distribution, these two quantiles would be ±1.96 for a 95% confidence interval.
For any pivot and confidence level, we can obtain numerical values for \(D_{\alpha / 2}\) and \(D_{1 - \alpha / 2}\)
Confidence interval
From how we defined these quantiles,
\[ P\left(D_{\alpha / 2} \;\lt\; g(\theta, X_1, \dots, X_n) \;\lt\; D_{1 - \alpha / 2} \right) \;\;=\;\; 1 - \alpha \]We can therefore define a \((1 - \alpha)\) confidence interval to be the values of \(\theta\) such that
\[ D_{\alpha / 2} \;\;\lt\;\; g(\theta, x_1, \dots, x_n) \;\;\lt\;\; D_{1 - \alpha / 2} \]Since there is a probability \((1 - \alpha)\) of this holding for a random sample from the distribution, the resulting confidence interval has confidence level \((1 - \alpha)\).