Target of small errors

Consider two possible sample statistics that could be used to estimate the centre of a symmetric population distribution — the sample mean and sample median. For each, there is an error,

error for mean  =   - µ

error for median  =  median - µ

The best estimator will be the one whose estimation erro is usually "closer to zero". This corresponds to two desirable characteristics of the error distribution.

Centred on zero

Ideally, we want the error distribution to be centred on zero. Such an estimator is called unbiased.

Sample means and proportions are unbiased estimators of the corresponding population parameters.

Small spread

Ideally, we also want error distribution to be tightly concentrated on zero — i.e. to have a small spread.

We call the standard deviation of the error distribution the standard error of the estimator. We ideally want an estimator with a small standard error.

standard error   =   standard deviation of the error

Note also that

standard error   =   standard deviation of the estimator

so a good estimator is one with a small standard deviation.