Sparse factor regression via penalized maximum likelihood estimation

Kei Hirose, Miyuki Imada

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

In factor regression model, the maximum likelihood estimation suffers from three disadvantages: (i) the maximum likelihood estimates are unavailable when the number of variables exceeds the number of observations, (ii) the rotation technique based on maximum likelihood estimates produces an insufficiently sparse loading matrix, and (iii) multicollinearity can occur when the estimates of unique variances (specific variances) are small because the regression coefficients are sensitive to the inverse of unique variances. To handle these problems, we propose a penalized maximum likelihood procedure. Specifically, we impose a lasso-type penalty on the factor loadings to improve the sparseness of the solution. We also introduce a penalty on unique variances, which (given the factor scores) corresponds to the ridge penalty on the regression coefficient. Theoretical properties from a prediction viewpoint of our procedure are discussed. The effectiveness of the procedure is investigated through Monte Carlo simulations. The utility of our procedure is demonstrated on real data collected by an online questionnaire.

Original languageEnglish
Pages (from-to)633-662
Number of pages30
JournalStatistical Papers
Volume59
Issue number2
DOIs
Publication statusPublished - Jun 1 2018

Fingerprint

Penalized Maximum Likelihood
Maximum Likelihood Estimation
Regression
Penalty
Regression Coefficient
Maximum Likelihood Estimate
Multicollinearity
Lasso
Factor Models
Ridge
Questionnaire
Regression Model
Exceed
Monte Carlo Simulation
Maximum likelihood estimation
Factors
Prediction
Estimate
Maximum likelihood
Coefficients

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Sparse factor regression via penalized maximum likelihood estimation. / Hirose, Kei; Imada, Miyuki.

In: Statistical Papers, Vol. 59, No. 2, 01.06.2018, p. 633-662.

Research output: Contribution to journalArticle

@article{936abbf04db140b69288521749a107bc,
title = "Sparse factor regression via penalized maximum likelihood estimation",
abstract = "In factor regression model, the maximum likelihood estimation suffers from three disadvantages: (i) the maximum likelihood estimates are unavailable when the number of variables exceeds the number of observations, (ii) the rotation technique based on maximum likelihood estimates produces an insufficiently sparse loading matrix, and (iii) multicollinearity can occur when the estimates of unique variances (specific variances) are small because the regression coefficients are sensitive to the inverse of unique variances. To handle these problems, we propose a penalized maximum likelihood procedure. Specifically, we impose a lasso-type penalty on the factor loadings to improve the sparseness of the solution. We also introduce a penalty on unique variances, which (given the factor scores) corresponds to the ridge penalty on the regression coefficient. Theoretical properties from a prediction viewpoint of our procedure are discussed. The effectiveness of the procedure is investigated through Monte Carlo simulations. The utility of our procedure is demonstrated on real data collected by an online questionnaire.",
author = "Kei Hirose and Miyuki Imada",
year = "2018",
month = "6",
day = "1",
doi = "10.1007/s00362-016-0781-8",
language = "English",
volume = "59",
pages = "633--662",
journal = "Statistical Papers",
issn = "0932-5026",
publisher = "Springer New York",
number = "2",

}

TY - JOUR

T1 - Sparse factor regression via penalized maximum likelihood estimation

AU - Hirose, Kei

AU - Imada, Miyuki

PY - 2018/6/1

Y1 - 2018/6/1

N2 - In factor regression model, the maximum likelihood estimation suffers from three disadvantages: (i) the maximum likelihood estimates are unavailable when the number of variables exceeds the number of observations, (ii) the rotation technique based on maximum likelihood estimates produces an insufficiently sparse loading matrix, and (iii) multicollinearity can occur when the estimates of unique variances (specific variances) are small because the regression coefficients are sensitive to the inverse of unique variances. To handle these problems, we propose a penalized maximum likelihood procedure. Specifically, we impose a lasso-type penalty on the factor loadings to improve the sparseness of the solution. We also introduce a penalty on unique variances, which (given the factor scores) corresponds to the ridge penalty on the regression coefficient. Theoretical properties from a prediction viewpoint of our procedure are discussed. The effectiveness of the procedure is investigated through Monte Carlo simulations. The utility of our procedure is demonstrated on real data collected by an online questionnaire.

AB - In factor regression model, the maximum likelihood estimation suffers from three disadvantages: (i) the maximum likelihood estimates are unavailable when the number of variables exceeds the number of observations, (ii) the rotation technique based on maximum likelihood estimates produces an insufficiently sparse loading matrix, and (iii) multicollinearity can occur when the estimates of unique variances (specific variances) are small because the regression coefficients are sensitive to the inverse of unique variances. To handle these problems, we propose a penalized maximum likelihood procedure. Specifically, we impose a lasso-type penalty on the factor loadings to improve the sparseness of the solution. We also introduce a penalty on unique variances, which (given the factor scores) corresponds to the ridge penalty on the regression coefficient. Theoretical properties from a prediction viewpoint of our procedure are discussed. The effectiveness of the procedure is investigated through Monte Carlo simulations. The utility of our procedure is demonstrated on real data collected by an online questionnaire.

UR - http://www.scopus.com/inward/record.url?scp=84969895684&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84969895684&partnerID=8YFLogxK

U2 - 10.1007/s00362-016-0781-8

DO - 10.1007/s00362-016-0781-8

M3 - Article

AN - SCOPUS:84969895684

VL - 59

SP - 633

EP - 662

JO - Statistical Papers

JF - Statistical Papers

SN - 0932-5026

IS - 2

ER -