Normal errors

Another assumption in the normal linear model is that the model errors are normally distributed.

ε  ~  normal (0 , σ)

If the model holds, the least squares residuals will also be normally distributed, so a histogram of the residuals can be examined for normality.

Normal errors are the least important of the model assumptions. If the other assumptions hold, it is reasonable to continue with the analysis, even if the errors have a skew distribution.

Normal probability plot

A better way to graphically examine a data set for normality is with a normal probability plot of the residuals. As with other probability plots, if the residuals are from a normal distribution, the crosses in the normal probability plot should lie close to a straight line.

How much curvature is needed to suggest non-normality?

In some data sets, linearity or nonlinearity in the probability plot is clear. In practice however, the randomness of real data means that the probability plot will not be exactly straight even for values that are sampled from a normal population.

How much curvature is needed to conclude that the underlying distribution is not normal?

This is a difficult question to answer. There are formal tests of normality that can be used in conjunction with a probability plot. (We discussed one in an earlier chapter about hypothesis tests.) We however take a less formal approach in the example below.

Deer jaw length and weight

Jaw lengths and body weights were recorded from 150 male deer that were shot in the Ruahine Ranges in New Zealand between November 1981 and November 1982. A scatterplot of the data and a probability plot of the residuals from the least squares line are shown below.

There is curvature at both ends of the probability plot, suggesting that the error distribution has longer tails than the normal distribution.

Could this amount of curvature have occurred by chance? Select Random Normal Data from the pop-up menu to generate random data from a normal linear model whose parameters β0, β1 and σ are the same as the least squares estimates from our data. Click Take Sample several times to see the variability in the probability plot when a normal linear model does hold.

The probability plot from the data seems more curved than the random ones, suggesting a problem with the assumption of normal errors.

Warning

If the assumptions of linearity and constant variance are violated, or if there are outliers, the probability plot of residuals will often be curved, irrespective of the error distribution.

Only draw a probability plot if you are sure that the data are linear, have constant variance and have no outliers.


Outliers — Deer data

In the Deer data above, most of the probabilitiy plot is fairly linear. The long tails to the residual distribution could perhaps be caused by a small number of outliers in the data.

If possible, it would be worth checking that there were no measurement or transcription errors for the most extreme residuals. Perhaps these measurements were made by one particular hunter who incorrectly measured jaw length?

Nonlinearity — Antibiotic effectiveness

Nonlinearity in the relationship can have the opposite effect on the shape of a probability plot — the tails of the distribution of residuals may be shorter than those for a normal distribution.

An experiment was conducted to assess how well an antibiotic, polymyxin B, killed the bacterium, Brucella bronchiseptica. Petri dishes containing the bacterium (grown in agar) were used and different doses of the antibiotic were added. The diameter of the area cleared around the addition point were recorded. Each of 6 different doses of antibiotic was used 10 times.

The diagram above shows the data and a probability plot of the residuals from the the least squares line. There is curvature in the probability plot, suggesting that the error distribution may have shorter tails than a normal distribution. (You can again select random samples where the normal linear model assumptions hold in order to help assess whether this amount of curvature in the probability plot is unusual.)

In this data set, curvature in the probability plot is probably caused by nonlinearity in the relationship. There is some indication that the relationship flattens out at high doses — perhaps a linear model with explanatory variable log(concentration) would be more linear?