Variance of the errors
A key assumption of the normal linear model is that all errors have the same variance,
All responses, yi, therefore also have variance σ2.
Variances of the residuals
The response can be written as,
Now since the variances of the responses and the fitted values are,
where hii is the leverage of the i'th observation, it can be proved that
(A full proof involves showing that the fitted values and residuals are uncorrelated.)
Therefore although the residuals from the least squares line, ei , can be considered to be estimates of the errors, their variances are all lower than σ2. Moreover,
The higher the leverage, the lower the residual variance.
Simulation
The diagram below demonstrates that all residuals do not have the same standard deviations. Response values are simulated from the model,
at x = 1.0, 3.5, 4.0, 4.5 and 5.0. The residuals from the least squares line are plotted against x on the right of the diagram.
Click Accumulate and take about 50 samples. Observe that the residuals have lower spread at x = 1.0 than at the other x-values, because of its higher leverage.
Choose Box plots and ± 2 sd from the pop-up menu to show the differences between the standard deviations of the residuals more clearly.
Theoretical distributions of the residuals
The diagram below shows the above regression model and the theoretical distributions of the residuals in rotating displays.