Minimization of Akaike's information criterion in linear regression analysis via mixed integer nonlinear program

Keiji Kimura, Hayato Waki

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Akaike's information criterion (AIC) is a measure of evaluating statistical models for a given data set. We can determine the best statistical model for a particular data set by finding the model with the smallest AIC value. Since there are exponentially many candidates of the best model, the computation of the AIC values for all the models is impractical. Instead, stepwise methods, which are local search algorithms, are commonly used to find a better statistical model, though it may not be the best model. We propose a branch-and-bound search algorithm for a mixed integer nonlinear programming formulation of the AIC minimization presented by Miyashiro and Takano [Mixed integer second-order cone programming formulations for variable selection, Eur. J. Oper. Res. 247 (2015), pp. 721–731]. More concretely, we propose procedures to find lower and upper bounds, and branching rules for this minimization. We then combine such procedures and branching rules with SCIP, a mathematical optimization software and the branch-and-bound framework. We show that the proposed method can provide the best AIC-based statistical model for small- or medium-sized benchmark data sets in the UCI Machine Learning Repository. Furthermore, the proposed method finds high-quality solutions for large-sized benchmark data sets.

Original languageEnglish
Pages (from-to)633-649
Number of pages17
JournalOptimization Methods and Software
Volume33
Issue number3
DOIs
Publication statusPublished - May 4 2018

Fingerprint

Akaike Information Criterion
Linear regression
Regression Analysis
Regression analysis
Statistical Model
Integer
Branching Rules
Benchmark
Second-order Cone Programming
Nonlinear programming
Mixed Integer Nonlinear Programming
Formulation
Local Search Algorithm
Learning systems
Cones
Branch and Bound Algorithm
Branch-and-bound
Variable Selection
Integer Programming
Model

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Optimization
  • Applied Mathematics

Cite this

Minimization of Akaike's information criterion in linear regression analysis via mixed integer nonlinear program. / Kimura, Keiji; Waki, Hayato.

In: Optimization Methods and Software, Vol. 33, No. 3, 04.05.2018, p. 633-649.

Research output: Contribution to journalArticle

@article{a7e171cff3b248fc981db9acebfe997a,
title = "Minimization of Akaike's information criterion in linear regression analysis via mixed integer nonlinear program",
abstract = "Akaike's information criterion (AIC) is a measure of evaluating statistical models for a given data set. We can determine the best statistical model for a particular data set by finding the model with the smallest AIC value. Since there are exponentially many candidates of the best model, the computation of the AIC values for all the models is impractical. Instead, stepwise methods, which are local search algorithms, are commonly used to find a better statistical model, though it may not be the best model. We propose a branch-and-bound search algorithm for a mixed integer nonlinear programming formulation of the AIC minimization presented by Miyashiro and Takano [Mixed integer second-order cone programming formulations for variable selection, Eur. J. Oper. Res. 247 (2015), pp. 721–731]. More concretely, we propose procedures to find lower and upper bounds, and branching rules for this minimization. We then combine such procedures and branching rules with SCIP, a mathematical optimization software and the branch-and-bound framework. We show that the proposed method can provide the best AIC-based statistical model for small- or medium-sized benchmark data sets in the UCI Machine Learning Repository. Furthermore, the proposed method finds high-quality solutions for large-sized benchmark data sets.",
author = "Keiji Kimura and Hayato Waki",
year = "2018",
month = "5",
day = "4",
doi = "10.1080/10556788.2017.1333611",
language = "English",
volume = "33",
pages = "633--649",
journal = "Optimization Methods and Software",
issn = "1055-6788",
publisher = "Taylor and Francis Ltd.",
number = "3",

}

TY - JOUR

T1 - Minimization of Akaike's information criterion in linear regression analysis via mixed integer nonlinear program

AU - Kimura, Keiji

AU - Waki, Hayato

PY - 2018/5/4

Y1 - 2018/5/4

N2 - Akaike's information criterion (AIC) is a measure of evaluating statistical models for a given data set. We can determine the best statistical model for a particular data set by finding the model with the smallest AIC value. Since there are exponentially many candidates of the best model, the computation of the AIC values for all the models is impractical. Instead, stepwise methods, which are local search algorithms, are commonly used to find a better statistical model, though it may not be the best model. We propose a branch-and-bound search algorithm for a mixed integer nonlinear programming formulation of the AIC minimization presented by Miyashiro and Takano [Mixed integer second-order cone programming formulations for variable selection, Eur. J. Oper. Res. 247 (2015), pp. 721–731]. More concretely, we propose procedures to find lower and upper bounds, and branching rules for this minimization. We then combine such procedures and branching rules with SCIP, a mathematical optimization software and the branch-and-bound framework. We show that the proposed method can provide the best AIC-based statistical model for small- or medium-sized benchmark data sets in the UCI Machine Learning Repository. Furthermore, the proposed method finds high-quality solutions for large-sized benchmark data sets.

AB - Akaike's information criterion (AIC) is a measure of evaluating statistical models for a given data set. We can determine the best statistical model for a particular data set by finding the model with the smallest AIC value. Since there are exponentially many candidates of the best model, the computation of the AIC values for all the models is impractical. Instead, stepwise methods, which are local search algorithms, are commonly used to find a better statistical model, though it may not be the best model. We propose a branch-and-bound search algorithm for a mixed integer nonlinear programming formulation of the AIC minimization presented by Miyashiro and Takano [Mixed integer second-order cone programming formulations for variable selection, Eur. J. Oper. Res. 247 (2015), pp. 721–731]. More concretely, we propose procedures to find lower and upper bounds, and branching rules for this minimization. We then combine such procedures and branching rules with SCIP, a mathematical optimization software and the branch-and-bound framework. We show that the proposed method can provide the best AIC-based statistical model for small- or medium-sized benchmark data sets in the UCI Machine Learning Repository. Furthermore, the proposed method finds high-quality solutions for large-sized benchmark data sets.

UR - http://www.scopus.com/inward/record.url?scp=85020536765&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020536765&partnerID=8YFLogxK

U2 - 10.1080/10556788.2017.1333611

DO - 10.1080/10556788.2017.1333611

M3 - Article

AN - SCOPUS:85020536765

VL - 33

SP - 633

EP - 649

JO - Optimization Methods and Software

JF - Optimization Methods and Software

SN - 1055-6788

IS - 3

ER -