Nonlinear regression with polynomials
We now extend quadratic models with extra terms involving higher powers of x.
The extra polynomial terms give considerable flexibility in modelling curvature in the relationship between Y and x. Since the parameters again appear linearly, this model is also a general linear model and all general results about GLMs apply to it.
Cubic model
The diagram below shows the X matrix that corresponds to a cubic model for a single explanatory variable, x.
Click any row to see how the 'linear part' of the model is a cubic in x.
Finding the best degree of polynomials
When using polynomial models, we are interested in using the lowest possible degree of polynomial — the lower the degree of polynomial, the smoother the fitted curve tends to become.
The test for whether the highest-order coefficients of a polynomial model is significantly different from zero can be used to decide whether to use a polynomial with lower degree.
For models of this type, we are interested in using the lowest possible degree of polynomial, but it makes no sense to have any power of x without all lower powers.
Note that it makes no sense to use a polynomial whose highest-order term is xp without including all lower powers in the model.
Do not therefore consider deletion of terms from the polynomial model if a higher-order term is still in the model.
Warning
A polynomial of degree 4 or higher might be found to fit the data best, but you will often find that it behaves in unreasonable ways outside the range of data that were collected. High-degree polynomials should never be used for extrapolation.
Some other form of nonlinear function should always be considered before using high-order polynomials.
For example, consider a nonlinear transformation of either X or Y.
Onion yield
The data below arose from an experiment in Purnong Landing, South Australia involving production of white Spanish onions. The explanatory variable is the density of planting (plants per square metre) and the response is onion yield (grams per plant).
The diagram initially shows a polynomial of degree 0 fitted by least squares to the data — simply the sample mean.
Use the pop-up menu to increase the polynomial degree. As extra polynomial terms are added to the model, the criterion of least squares ensures that the curve becomes closer to the data — the residual sum of squares reduces.
The polynomial is fairly smooth for all degrees up to 5. However observe that the polynomial of degree 5 has a sharp increase for densities over 180 and would be a poor predictor of yield outside the range of the data (densities of 20 to 180).
Traffic fatal crashes in New Zealand
The next data set gives the number of fatal vehicle crashes in New Zealand per 100,000 population between 1970 and 2005.
Again examine how the shape of the least squares curve changes as the polynomial degree increases.
Several of the polynomials seem reasonable descriptions of the changes in fatal vehicle crashes between 1970 and 2005. However none of the polynomials would provide reasonable predictions after 2005.