Parameter estimation

Since the residuals can be considered to be prediction errors, a good linear equation should result in small residuals.

A good principle is therefore to estimate the three parameters of the model (the intercept and the two slope parameters) by adjusting them to make the residuals 'as small as possible'.

Illustration

The diagram below shows an artificial data set.

Drag the three red arrows vertically to adjust the three parameters of the model. Find parameter estimates to make the residuals as small as possible.

Hint: It can be tricky getting the residuals small for all combinations of X and Z. Adjusting the parameters in the following order helps.

  1. Firstly adjust the intercept to give small residuals when X and Z are close to zero.
  2. Next, adjust the slope for X to make the residuals small for all crosses where Z is close to zero.
  3. Finally, adjust the slope for Z to make all residuals small.