Constructs a regression tree (R.W. Payne).
|Controls printed output (
||Response variate for the regression|
||Saves the tree that has been constructed|
||Limit on the mean square of the observations at a node at which to stop making splits; default 0|
||Specifies the number of observations at a node at which to stop making splits; default 1|
||Indicates whether or not your own version of the
||Independent variables available for constructing the tree|
||Whether factor levels are ordered (
A regression tree is a mechanism for predicting a response variable from a set of independent variables (see Chapter 8 of Breiman et al.). The tree is constructed using data on a set of observations. Their values for the response variable are specified (in a variate) using the
Y option, and their values for the independent variables are specified (in a list of variates or factors) using the
X parameter. Factors may have either ordered or unordered levels, according to whether the corresponding value
ORDERED parameter is set to
no. For example, a factor called
Dose with levels 1, 1.5, 2 and 2.5 would usually be treated as having ordered levels, whereas levels labelled
'Pethidine' of a factor called
Drug would be regarded as unordered.
The construction process splits the observations into subsets. With an x-variate or a factor with ordered levels, the subsets are formed by taking the observations with values less then or greater than some split point p. For a factor with unordered levels, all possible ways of dividing its levels into two subsets are tried. The aim is to form subsets that have similar values for the response variate. The predicted value of the response variable for each node of the tree is the mean of its value for the subset of observations at that node. The accuracy of the node is the squared distance of the values of the response variate from their mean for the observations at the node, divided by the total number of observations. The potential splits at the node are assessed by their effect on the accuracy, that is the difference between the accuracy of the node and the sum of the accuracies of the two potential successor nodes. The node will become a terminal node if none of the splits provides any improvement in accuracy, or if the mean square of the observations at the node is less than or equal to a limit specified by the
MSLIMIT option (default 0), or if the number of observations at the node is less than or equal to the number specified by the
NSTOP option (default 1).
The resulting tree can be saved using the
TREE option. Details of the tree can be printed as selected by the
||prints a summary of the properties of the tree;|
||gives detailed information about the nodes of the tree;|
||display as used to represent an identification key in “bracketed” form (printed node by node).|
||display as used to represent an identification key in “indented” form (printed branch by branch);|
||diagrammatic display including the node labels;|
||diagrammatic display with the nodes labelled by their numbers;|
||plots the tree using high-resolution graphics.|
||prints information monitoring the construction process.|
BREGRESSION stores the information required for printing as part of the tree. For variates and ordered factors, the labels are generally formed as “identifier<p” and “identifier>p“, where p is the value chosen to partition the data for the variate concerned. Alternatively, if you have defined an “extra” text for the variate (using the
EXTRA parameter of the
VARIATE command), this will be used instead. The labels are then “extra-text < p” and “extra-text > p“. The style is similar for unordered factors, but here the labels involve the operators
.NI. instead of .
Generally the construction will result in over-fitting, that is it will form a tree that keeps making splits beyond the point that can be justified statistically. The solution is to prune the tree to remove the uninformative sub-branches, and this can be performed using the
BPRUNE procedure. It is best, if possible, to base the pruning on an independent set of data. The pruning uses the accuracy figures, which are stored with the tree. The
BRVALUES procedure can be used to calculate new accuracy (and prediction) values, from another data set.
Finally, once the tree has been pruned, the value predicted for a new set of independent values can be obtained by supplying their values to the
BRPREDICT procedure. This runs the values through the tree to see which terminal node they reach. The prediction is then provided by the value predicted at that node.
BREGRESSION calls procedure
BCONSTRUCT to form the tree. This uses a special-purpose procedure
BSELECT, which is customized specifically to select splits for use in regression trees and stored with
BREGRESSION. You can use your own method of selection by providing your own
BSELECT and setting option
OWNBSELECT=yes. In the standard version of
BASSESS directive is used to assess the potential splits.
Any restrictions on the
X variates are removed.
Breiman, L., Friedman, J.H., Olshen, R.A. & Stone, C.J. (1984). Classification and Regression Trees. Wadsworth, Monterey.
CAPTION 'BREGRESSION example',!t('Water usage data (Draper & Smith 1981,',\ 'Applied Regression Analysis, Wiley, New York).'); STYLE=meta,plain READ temp,product,opdays,employ,water 58.8 7.107 21 129 3.067 65.2 6.373 22 141 2.828 70.9 6.796 22 153 2.891 77.4 9.208 20 166 2.994 79.3 14.792 25 193 3.082 81.0 14.564 23 189 3.898 71.9 11.964 20 175 3.502 63.9 13.526 23 186 3.060 54.5 12.656 20 190 3.211 39.5 14.119 20 187 3.286 44.5 16.691 22 195 3.542 43.6 14.571 19 206 3.125 56.0 13.619 22 198 3.022 64.7 14.575 22 192 2.922 73.0 14.556 21 191 3.950 78.9 18.573 21 200 4.488 79.4 15.618 22 200 3.295 : "form the regression tree" BREGRESSION [PRINT=summary,indented; Y=water; TREE=tree]\ employ,opdays,product,temp "prune the tree" BPRUNE [PRINT=table,graph] tree; NEWTREES=pruned "use tree 6 - renumber nodes" BCUT [RENUMBER=yes] pruned; NEWTREE=tree "display the tree" BRDISPLAY [PRINT=summary,indented,graph] tree