Describing accuracy with an interval estimate

Maximum likelihood provides a single value called a point estimate, \(\hat{\theta}\), of a parameter \(\theta\). However this does not convey any information about how close to \(\theta\) this estimate is likely to be. We need to find a way to describe the estimator's accuracy — i.e. the likely size of the estimation error.

For example, it is more informative to make a statement such as

" \(\theta\) is probably between 10.5 and 11.5"

rather than simply

"Our best estimate of \(\theta\) is 11.0"

This is called an interval estimate of \(\theta\).

Will the interval estimate include \(\theta\)?

We can rarely be certain that an interval estimate will include the actual parameter value, \(\theta\). The narrower the interval, the less certain we will be that it will contain \(\theta\).

Estimating a distribution's mean, μ

For example, we might estimate a distribution's mean, \(\mu\), using the mean of a random sample, \(\overline x\). The diagram below shows five possible intervals that might be used to describe the likely value of \(\mu\), based on a random sample of \(n=10\) values.

As the width of the interval estimate is reduced, we become less confident that it will include the unknown value, \(\mu\).

We need to find a way to quantify our confidence that any particular interval estimate will include the parameter being estimated.