Model

Consider data that come from a general linear model,

in which the errors, εi, are independent and

Estimate of error variance

The general linear model involves a further parameter whose value is unknown and must be estimated — the error standard deviation, σ. To simplify the formulae, we will estimate its square, the error variance, σ2.

The best estimate of σ2 is the residual sum of squares, divided by its degrees of freedom, called the mean residual sum of squares. Its degrees of freedom are the number of observations, n, minus the number of columns of X.

Note that p is the number of explanatory variables plus 1 because of the column of 1's in X.

Body fat

Fitting a general linear model to the body fat data gives the following table of least squares estimates.

The diagram shows the sum of squared residuals from the model and its degrees of freedom (n = 252 observations minus p = 14 estimated parameters in b ). The estimate of the error standard deviation is also given — it is about 4.0 percent body fat.

When estimating a man's percentage body fat from the 13 recorded variables, there is an unavoidable and unpredictable normal error with standard deviation about 4.0 percent body fat.

There is about 95% probability that the error will be within 8 percent body fat.


Standard errors of least squares estimates

As in other situations where a parameter is estimated, it is important to give standard errors for the least squares estimates, b0, b1, ..., of the model parameters. These are needed in order to find confidence intervals for the individual slope parameters and for hypothesis tests about their values.

We will give formulae for these standard errors below, but in practice a computer can be relied on to evaluate them.

Body fat

The computer can obtain standard errors for the parameters of the body fat model. Click the checkbox Show std errors in the table above to see the standard errors of the least squares estimates.

These standard errors describe the accuracy of the least squares estimates.


Formula for variances of least squares estimates (optional)

The matrix below gives the variance-covariance matrix of the least squares estimates.

The diagonal elements of this matrix are the variances of the individual parameter estimates.

(The off-diagonal elements of this matrix are related to the correlations between pairs of parameter estimates — they are often correlated — but we will not use them here.)

Standard errors of least squares estimates (optional)

This formula involves the unknown error variance, σ2. Replacing it with the mean residual sum of squares, we can obtain a numerical estimate of the variance-covariance matrix,

The standard deviations of the least squares coefficients (i.e. their standard errors when we think of them as estimates of the elements of ) are the square roots of the diagonal elements of this matrix.

You will not need to use these matrix formulae yourself — computer software will find the values for you.

It is however important to recognise that:

Parameter estimates and their standard errors can be expressed with the same simple matrix formulae for all general linear models.