Interpreting a confidence interval
It is important to remember that a confidence interval does not always "work" — it may not actually include the unknown parameter value. For example, a 95% confidence interval might be
\[ 0.7773 \;\;\lt\;\; \theta \;\;\lt\;\; 0.9483 \]The value of \(\theta\) is unknown and may not actually lie within this interval. The best we can say is that we are 95% confident that \(\theta\) will be between these two values.
If confidence intervals were found from other similar random samples, 95% of them would include \(\theta\).
The notion of a confidence level therefore more a characteristic of the method of finding the confidence interval, than a characteristic of the specific confidence interval from a single data set.
Illustration
In the following diagram, 200 success/failure values have been randomly selected with probability \(\pi = 0.6\) of success. The number of successes therefore has a binomial distribution.
We will pretend that the value of \(\pi\) is unknown and find a 95% confidence interval for it, as described earlier in this section. This confidence interval is shown graphically with a horizontal line on the right of the diagram.
Click Take sample a few times to collect other samples of 200 success/failure values and observe that the 95% confidence intervals vary from sample to sample.
Next, click the checkbox Accumulate and take more samples to accumulate a display of 100 or more confidence intervals on the right of the diagram. Observe that approximately 95% of these intervals include the population probability of success, \(\pi = 0.6\).
The confidence interval from any single data set may or may not include \(\pi\) and, in practice, we don't know whether it will be one of the 95% of "lucky" ones. The best we can say is that we are 95% confident that it will.