A note on model selection for small sample regression

Masanori Kawakita, Junnichi Takeuchi

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

The risk estimator called “Direct Eigenvalue Estimator” (DEE) is studied. DEE was developed for small sample regression. In contrast to many existing model selection criteria, derivation of DEE requires neither any asymptotic assumption nor any prior knowledge about the noise variance and the noise distribution. It was reported that DEE performed well in small sample cases but DEE performed a little worse than the state-of-the-art ADJ. This seems somewhat counter-intuitive because DEE was developed for specifically regression problem by exploiting available information exhaustively, while ADJ was developed for general setting. In this paper, we point out that the derivation of DEE includes an inappropriate part, notwithstanding the resultant form of DEE being valid in a sense. As its result, DEE cannot derive its potential. We introduce a class of ‘valid’ risk estimators based on the idea of DEE and show that better risk estimators (mDEE) can be found in the class. By numerical experiments, we verify that mDEE often performs better than or at least equally the original DEE and ADJ.

Original languageEnglish
Pages (from-to)1839-1862
Number of pages24
JournalMachine Learning
Volume106
Issue number11
DOIs
Publication statusPublished - Nov 1 2017

Fingerprint

Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Cite this

A note on model selection for small sample regression. / Kawakita, Masanori; Takeuchi, Junnichi.

In: Machine Learning, Vol. 106, No. 11, 01.11.2017, p. 1839-1862.

Research output: Contribution to journalArticle

Kawakita, Masanori ; Takeuchi, Junnichi. / A note on model selection for small sample regression. In: Machine Learning. 2017 ; Vol. 106, No. 11. pp. 1839-1862.
@article{5b4a8896476c470093317a082ccb1914,
title = "A note on model selection for small sample regression",
abstract = "The risk estimator called “Direct Eigenvalue Estimator” (DEE) is studied. DEE was developed for small sample regression. In contrast to many existing model selection criteria, derivation of DEE requires neither any asymptotic assumption nor any prior knowledge about the noise variance and the noise distribution. It was reported that DEE performed well in small sample cases but DEE performed a little worse than the state-of-the-art ADJ. This seems somewhat counter-intuitive because DEE was developed for specifically regression problem by exploiting available information exhaustively, while ADJ was developed for general setting. In this paper, we point out that the derivation of DEE includes an inappropriate part, notwithstanding the resultant form of DEE being valid in a sense. As its result, DEE cannot derive its potential. We introduce a class of ‘valid’ risk estimators based on the idea of DEE and show that better risk estimators (mDEE) can be found in the class. By numerical experiments, we verify that mDEE often performs better than or at least equally the original DEE and ADJ.",
author = "Masanori Kawakita and Junnichi Takeuchi",
year = "2017",
month = "11",
day = "1",
doi = "10.1007/s10994-017-5645-5",
language = "English",
volume = "106",
pages = "1839--1862",
journal = "Machine Learning",
issn = "0885-6125",
publisher = "Springer Netherlands",
number = "11",

}

TY - JOUR

T1 - A note on model selection for small sample regression

AU - Kawakita, Masanori

AU - Takeuchi, Junnichi

PY - 2017/11/1

Y1 - 2017/11/1

N2 - The risk estimator called “Direct Eigenvalue Estimator” (DEE) is studied. DEE was developed for small sample regression. In contrast to many existing model selection criteria, derivation of DEE requires neither any asymptotic assumption nor any prior knowledge about the noise variance and the noise distribution. It was reported that DEE performed well in small sample cases but DEE performed a little worse than the state-of-the-art ADJ. This seems somewhat counter-intuitive because DEE was developed for specifically regression problem by exploiting available information exhaustively, while ADJ was developed for general setting. In this paper, we point out that the derivation of DEE includes an inappropriate part, notwithstanding the resultant form of DEE being valid in a sense. As its result, DEE cannot derive its potential. We introduce a class of ‘valid’ risk estimators based on the idea of DEE and show that better risk estimators (mDEE) can be found in the class. By numerical experiments, we verify that mDEE often performs better than or at least equally the original DEE and ADJ.

AB - The risk estimator called “Direct Eigenvalue Estimator” (DEE) is studied. DEE was developed for small sample regression. In contrast to many existing model selection criteria, derivation of DEE requires neither any asymptotic assumption nor any prior knowledge about the noise variance and the noise distribution. It was reported that DEE performed well in small sample cases but DEE performed a little worse than the state-of-the-art ADJ. This seems somewhat counter-intuitive because DEE was developed for specifically regression problem by exploiting available information exhaustively, while ADJ was developed for general setting. In this paper, we point out that the derivation of DEE includes an inappropriate part, notwithstanding the resultant form of DEE being valid in a sense. As its result, DEE cannot derive its potential. We introduce a class of ‘valid’ risk estimators based on the idea of DEE and show that better risk estimators (mDEE) can be found in the class. By numerical experiments, we verify that mDEE often performs better than or at least equally the original DEE and ADJ.

UR - http://www.scopus.com/inward/record.url?scp=85021069972&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021069972&partnerID=8YFLogxK

U2 - 10.1007/s10994-017-5645-5

DO - 10.1007/s10994-017-5645-5

M3 - Article

AN - SCOPUS:85021069972

VL - 106

SP - 1839

EP - 1862

JO - Machine Learning

JF - Machine Learning

SN - 0885-6125

IS - 11

ER -