# Glmnet in Matlab

### Description

This is a Matlab port for the extremely efficient procedures for fitting the entire lasso or elastic-net regularization path for linear regression, logistic and multinomial regression models, poisson regression and the Cox model. Two recent additions are the multiresponse gaussian, and the grouped multinomial.

Suppose $X$ is the input matrix and $Y$ the response vector. For the Gaussian family, glmnet solves the penalized residual sum of squares,

$$\min_{(\beta_0, \beta) \in \mathbb{R}^{p+1}}\frac{1}{2N} \sum_{i=1}^N (y_i -\beta_0-x_i^T \beta)^2+\lambda \big[ (1-\alpha)||\beta||_2^2/2 + \alpha||\beta||_1\big],$$

where $\lambda \geq 0$ is a complexity parameter and $0 \leq \alpha \leq 1$ is a compromise between ridge and lasso. Note that it becomes the lasso when $\alpha = 1$ and the ridge regression when $\alpha = 0$.

For other families, glmnet maximizes the appropriate penalized log-likelihood (partial likelihood for the "cox" model), or minimize the penalized negative one. Take the "binomial" model for example, it solves

$$\min_{(\beta_0, \beta) \in \mathbb{R}^{p+1}} -\frac{1}{N} \sum_{i=1}^N y_i \cdot (\beta_0 + x_i^T \beta) + \log (1+e^{(\beta_0+x_i^T \beta)}) + \lambda \big[ (1-\alpha)||\beta||_2^2/2 + \alpha||\beta||_1\big].$$

The algorithm uses cyclical coordinate descent in a pathwise fashion. In addition to basic settings, many more options are available: observation weights, choice of lambda sequence, grouping, etc. For more information, see the reference papers, help file or the documentation (in progress).

The package includes two main functions:

• glmnet.m - the basic function that returns a structure containing all necessary information for further use, like printing, plotting and prediction.
• cvglmnet.m - a more commonly used function that returns a structure after selecting the tuning parameter by cross-validation according to user's requirements.

A basic example is that suppose x is the design matrix and y the response vector, then,

• fit = glmnet(x, y) -- fits the model using all default settings.
• glmnetPrint(fit) -- prints relevant information of the fitted object.
• glmnetPlot(fit) -- plots the coefficients from the fitted object.
• cvfit = cvglmnet(x, y) -- fits the model by cross-validation using all default settings.
• cvglmnetPlot(cvfit) -- plots the cross-validation curve.

For more details and examples, enter help glmnet or help cvglmnet in the Matlab's command window. Other functions are listed below.

 cvglmnet.m Cross-validation for glmnet cvglmnetCoef.m Extract the coefficients from a 'cv.glmnet' object cvglmnetPlot.m Plot the cross-validation curve produced by cvglmnet.m cvglmnetPredict.m Make predictions from a 'cv.glmnet' object glmnet.m Fit a GLM with lasso or elasticnet regularization glmnetCoef.m Extract the coefficients from a 'glmnet' object glmnetControl.m Internal glmnet parameters glmnetPlot.m Plot coefficients from a 'glmnet' object glmnetPredict.m Make predictions from a 'glmnet' object glmnetPrint.m Print a 'glmnet' object

### Authors

Algorithms designed by Jerome Friedman, Trevor Hastie, Noah Simon and Rob Tibshirani. Fortran code was written by Jerome Friedman.
The original MATLAB wrapper was written by Hui Jiang (14 Jul 2009). It was updated (30 Aug 2013) and is maintained by Junyang Qian junyangq@stanford.edu, Department of Statistics, Stanford University, Stanford, California, USA.

The package currently supports the following platforms.

We will add support for other platforms such as Windows 7. Users can also compile the code on their platforms easily. On 32-bit systems, users can simply execute the mex function

mex glmnetMex.F GLMnet.f

On 64-bit systems, compiler options should be used to make it double precision. For example, on the 64-bit Mac OS X, we used

mex FFLAGS='-fdefault-real-8 -ffixed-form' glmnetMex.F GLMnet.f

If the compile is successful, we are ready to go!

### References

Jerome Friedman, Trevor Hastie and Rob Tibshirani. (2008).
Regularization Paths for Generalized Linear Models via Coordinate Descent
Journal of Statistical Software, Vol. 33(1), 1-22 Feb 2010.

Noah Simon, Jerome Friedman, Trevor Hastie and Rob Tibshirani. (2011).
Regularization Paths for Cox's Proportional Hazards Model via Coordinate Descent
Journal of Statistical Software, Vol. 39(5) 1-13.

Robert Tibshirani, Jacob Bien, Jerome Friedman, Trevor Hastie, Noah Simon, Jonathan Taylor, Ryan J. Tibshirani. (2010).
Strong Rules for Discarding Predictors in Lasso-type Problems
Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(2), 245-266.