TY - GEN
T1 - Simultaneous safe screening of features and samples in doubly sparse modeling
AU - Shibagaki, Atsushi
AU - Karasuyama, Masayuki
AU - Hatano, Kohei
AU - Takeuchi, Ichiro
N1 - Funding Information:
For this work, MK was partially supported by JSPS KAK- ENHI Grant Number 26280083. KH was partially supported by JSPS KAKENHI Grant Number 16K00305 and MEXT KAKENHI Grant Number 24106010 (the ELC project). IT was partially supported by JST CREST 13217726, CREST 15656320, JSPS KAKENHI Grant Number 26280083, MEXT KAKENHI Grant Number 16H00886, and JST support program for starting up innovation-hub on materials research by information inte-gration initiative.
Publisher Copyright:
© 2016 by the author(s).
PY - 2016
Y1 - 2016
N2 - The problem of learning a sparse model is conceptually interpreted as the process of identify-ing active features/samples and then optimizing the model over them. Recently introduced safe screening allows us to identify a part of non- Active features/samples. So far, safe screening has been individually studied either for feature screening or for sample screening. In this paper, we introduce a new approach for safely screening features and samples simultaneously by al-ternatively iterating feature and sample screening steps. A significant advantage of considering them simultaneously rather than individually is that they have a synergy effect in the sense that the results of the previous safe feature screening can be exploited for improving the next safe sample screening performances, and vice- versa. We first theoretically investigate the synergy effect, and then illustrate the practical advantage through intensive numerical experiments for problems with large numbers of features and samples.
AB - The problem of learning a sparse model is conceptually interpreted as the process of identify-ing active features/samples and then optimizing the model over them. Recently introduced safe screening allows us to identify a part of non- Active features/samples. So far, safe screening has been individually studied either for feature screening or for sample screening. In this paper, we introduce a new approach for safely screening features and samples simultaneously by al-ternatively iterating feature and sample screening steps. A significant advantage of considering them simultaneously rather than individually is that they have a synergy effect in the sense that the results of the previous safe feature screening can be exploited for improving the next safe sample screening performances, and vice- versa. We first theoretically investigate the synergy effect, and then illustrate the practical advantage through intensive numerical experiments for problems with large numbers of features and samples.
UR - http://www.scopus.com/inward/record.url?scp=84998693042&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84998693042&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84998693042
T3 - 33rd International Conference on Machine Learning, ICML 2016
SP - 2381
EP - 2390
BT - 33rd International Conference on Machine Learning, ICML 2016
A2 - Weinberger, Kilian Q.
A2 - Balcan, Maria Florina
PB - International Machine Learning Society (IMLS)
T2 - 33rd International Conference on Machine Learning, ICML 2016
Y2 - 19 June 2016 through 24 June 2016
ER -