Tuning parameter selection in sparse regression modeling

Kei Hirose, Shohei Tateishi, Sadanori Konishi

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' Cp type criteria may be used as a tuning parameter selection tool in lasso type regularization methods, for which the concept of degrees of freedom plays a key role. In the present paper, we propose an efficient algorithm that computes the degrees of freedom by extending the generalized path seeking algorithm. Our procedure allows us to construct model selection criteria for evaluating models estimated by regularization with a wide variety of convex and nonconvex penalties. The proposed methodology is investigated through the analysis of real data and Monte Carlo simulations. Numerical results show that Cp criterion based on our algorithm performs well in various situations.

Original languageEnglish
Pages (from-to)28-40
Number of pages13
JournalComputational Statistics and Data Analysis
Volume59
Issue number1
DOIs
Publication statusPublished - Mar 1 2013
Externally publishedYes

Fingerprint

Lasso
Parameter Selection
Parameter Tuning
Tuning
Regularization
Regression
Degree of freedom
Model Evaluation
Model Selection Criteria
Regularization Parameter
Regularization Method
Modeling
Model Selection
Penalty
Efficient Algorithms
Monte Carlo Simulation
Numerical Results
Path
Methodology
Model

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Computational Mathematics
  • Computational Theory and Mathematics
  • Applied Mathematics

Cite this

Tuning parameter selection in sparse regression modeling. / Hirose, Kei; Tateishi, Shohei; Konishi, Sadanori.

In: Computational Statistics and Data Analysis, Vol. 59, No. 1, 01.03.2013, p. 28-40.

Research output: Contribution to journalArticle

Hirose, Kei ; Tateishi, Shohei ; Konishi, Sadanori. / Tuning parameter selection in sparse regression modeling. In: Computational Statistics and Data Analysis. 2013 ; Vol. 59, No. 1. pp. 28-40.
@article{2cced5ac682e45658f768c1f4903d25c,
title = "Tuning parameter selection in sparse regression modeling",
abstract = "In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' Cp type criteria may be used as a tuning parameter selection tool in lasso type regularization methods, for which the concept of degrees of freedom plays a key role. In the present paper, we propose an efficient algorithm that computes the degrees of freedom by extending the generalized path seeking algorithm. Our procedure allows us to construct model selection criteria for evaluating models estimated by regularization with a wide variety of convex and nonconvex penalties. The proposed methodology is investigated through the analysis of real data and Monte Carlo simulations. Numerical results show that Cp criterion based on our algorithm performs well in various situations.",
author = "Kei Hirose and Shohei Tateishi and Sadanori Konishi",
year = "2013",
month = "3",
day = "1",
doi = "10.1016/j.csda.2012.10.005",
language = "English",
volume = "59",
pages = "28--40",
journal = "Computational Statistics and Data Analysis",
issn = "0167-9473",
publisher = "Elsevier",
number = "1",

}

TY - JOUR

T1 - Tuning parameter selection in sparse regression modeling

AU - Hirose, Kei

AU - Tateishi, Shohei

AU - Konishi, Sadanori

PY - 2013/3/1

Y1 - 2013/3/1

N2 - In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' Cp type criteria may be used as a tuning parameter selection tool in lasso type regularization methods, for which the concept of degrees of freedom plays a key role. In the present paper, we propose an efficient algorithm that computes the degrees of freedom by extending the generalized path seeking algorithm. Our procedure allows us to construct model selection criteria for evaluating models estimated by regularization with a wide variety of convex and nonconvex penalties. The proposed methodology is investigated through the analysis of real data and Monte Carlo simulations. Numerical results show that Cp criterion based on our algorithm performs well in various situations.

AB - In sparse regression modeling via regularization such as the lasso, it is important to select appropriate values of tuning parameters including regularization parameters. The choice of tuning parameters can be viewed as a model selection and evaluation problem. Mallows' Cp type criteria may be used as a tuning parameter selection tool in lasso type regularization methods, for which the concept of degrees of freedom plays a key role. In the present paper, we propose an efficient algorithm that computes the degrees of freedom by extending the generalized path seeking algorithm. Our procedure allows us to construct model selection criteria for evaluating models estimated by regularization with a wide variety of convex and nonconvex penalties. The proposed methodology is investigated through the analysis of real data and Monte Carlo simulations. Numerical results show that Cp criterion based on our algorithm performs well in various situations.

UR - http://www.scopus.com/inward/record.url?scp=84870055613&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84870055613&partnerID=8YFLogxK

U2 - 10.1016/j.csda.2012.10.005

DO - 10.1016/j.csda.2012.10.005

M3 - Article

VL - 59

SP - 28

EP - 40

JO - Computational Statistics and Data Analysis

JF - Computational Statistics and Data Analysis

SN - 0167-9473

IS - 1

ER -