Performs lasso using iteratively reweighted least-squares (D.A. Murray & P.H.C. Eilers).
|What output to print (
||What graphs to plot (
||Limit on number of factors/covariates in a model term; default 3|
||Values for the parameter lambda; must be set|
||Which cross-validation method to use (
||Number of groups for k-fold cross-validation; default 10|
||Seed for random numbers to use in cross-validation; default 0|
||Maximum number of iterations for the iterative process|
||Contains two values to define the convergence criterion for iterative least-squares and the adjustment to avoid division by zero in the penalty term; default
||Saves the optimal lambda value from cross-validation|
||Saves the cross-validation statistics|
||Saves residuals for the optimal
||Saves fitted values for the optimal
||Saves parameter estimates for the optimal
||Saves standard errors of the parameter estimates for the optimal
RLASSO procedure performs L1-penalized regression (lasso) using iteratively reweighted sums of squares. The lasso method minimizes the residual sums of squares subject to the constraint that the sum of the absolute values of the model coefficients is less than a constant or tuning parameter λ.
The response variate is specified by the
Y parameter. The model to be fitted is defined by the
TERMS option. The
FACTORIAL option sets a limit on the number of variates and/or factors in the model terms generated from the
TERMS model formula (as in the
Printed output is controlled by the
||to print the correlations between the explanatory variables in the
||to print the cross-validation results, with optimal lambda value,|
||shows the progress of the k-fold cross-validation,,|
||prints the lasso estimates for the optimal λ, and|
||to print, for each value of λ, the lasso coefficients their standard errors on the standardized and original scales.|
Graphical output is controlled by the
||plots the standardized coefficient estimates against the shrinkage factor, and correlation, and|
By default, nothing is plotted.
LAMBDA option must be set to a variate defining the values to try for the tuning parameter λ. The
MAXCYCLE option specifies the number of iterations (default 200). The
TOLERANCE option specifies the convergence criterion for the iterative procedure (default 0.0001), and the adjustment to use to avoid division by zero in the penalty term (default 10-8).
VALIDATIONMETHOD option controls how
RLASSO estimates the tuning parameter λ:
||uses k-fold cross-validation where the prediction error is calculated using the mean squared error,|
||uses the generalized cross-validation, as specified by Tibshirani (1996).|
By default ,
For k-fold cross-validation the
NCROSSVALIDATIONGROUPS option defines the number of subsets to use (default 10). The data are divided into roughly equal-sized subsets and the model is fitted with each subset removed in turn. The mean squared error is calculated for the omitted subset based on the model from fitting the remaining subsets. The value that minimizes the mean prediction error is taken as the optimal λ, and used to get the lasso estimates. The optimal value of λ can be saved by the
BESTLAMBDA parameter, and the prediction error values can be saved by the
You can save results from the optimal fit using the
SE parameters. Note that the residuals are the simple residuals, rather than standardized residuals.
Lasso is carried out by using iteratively reweighted least-squares.
RLASSO approximates the absolute sum of the coefficients ∑|β| by ∑(β2/|β|), and the penalty term λ∑(β2/|β|) is imposed on the sum of squares of the parameter estimates β. The penalty term is applied to the diagonal elements of the sums-of-squares-and-products matrix by setting the
RIDGE option of the
TERMS directive. For a given value of λ, the algorithm iterates to find the lasso estimates. The shrinkage factor s is estimated by
s = t / ∑|β(0)|
where ∑|β(0)| is the absolute sum of the full least squares estimates, and t is the absolute sum of the lasso estimates subject to
t ≤ ∑|β(0)|.
The columns of the design matrix in
TERMS are standardized. However, estimated coefficients are available for both the standardized and unstandardized data.
There must be no restrictions.
Hastie, T., Tibshirani, R. & Friedman, J (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd Edition. Springer, New York.
Tibshirani, R. (1996). Regression shrinkage and selection by lasso. Journal of the Royal Statistical Society, Series B, 58, 267-288.
Commands for: Regression analysis.
CAPTION 'RLASSO example'; STYLE=meta " Prostate cancer data examining the correlation between the level of prostate-specific antigen and some clinical measures. See Tibshirani (1996), Regression and Selection by Lasso, JRSS B, 58, 267-288." SPLOAD '%GENDIR%/Examples/RLAS-1.gsh' SUBSET [train.eq.2] lcavol,lweight,age,lbph,svi,lcp,gleason,pgg45,lpsa CALCULATE lambdas = 10**(!(1.8,1.7...-2)) RLASSO [PRINT=correlation,estimates,cross,best;\ PLOT=coefficients,correlation; LAMBDA=lambdas;\ TERMS=lcavol,lweight,age,lbph,svi,lcp,gleason,pgg45]\ Y=lpsa; BEST=optlambda; ESTIMATES=estimates; SE=se PRINT optlambda PRINT estimates,se