Interpreting a confidence interval

It is important to remember that a confidence interval does not always "work" — it may not actually include the unknown parameter value. For example, a 95% confidence interval might be

\[ 0.7773 \;\;\lt\;\; \theta \;\;\lt\;\; 0.9483 \]

The value of \(\theta\) is unknown and may not actually lie within this interval. The best we can say is that we are 95% confident that \(\theta\) will be between these two values.

If confidence intervals were found from other similar random samples, 95% of them would include \(\theta\).

The notion of a confidence level therefore more a characteristic of the method of finding the confidence interval, than a characteristic of the specific confidence interval from a single data set.

Illustration

In the following diagram, 200 success/failure values have been randomly selected with probability \(\pi = 0.6\) of success. The number of successes therefore has a binomial distribution.

We will pretend that the value of \(\pi\) is unknown and find a 95% confidence interval for it, as described earlier in this section. This confidence interval is shown graphically with a horizontal line on the right of the diagram.

Click Take sample a few times to collect other samples of 200 success/failure values and observe that the 95% confidence intervals vary from sample to sample.

Next, click the checkbox Accumulate and take more samples to accumulate a display of 100 or more confidence intervals on the right of the diagram. Observe that approximately 95% of these intervals include the population probability of success, \(\pi = 0.6\).

The confidence interval from any single data set may or may not include \(\pi\) and, in practice, we don't know whether it will be one of the 95% of "lucky" ones. The best we can say is that we are 95% confident that it will.