Higher order fused regularization for supervised learning with grouped parameters

Koh Takeuchi, Yoshinobu Kawahara, Tomoharu Iwata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

We often encounter situations in supervised learning where there exist possibly groups that consist of more than two parameters. For example, we might work on parameters that correspond to words expressing the same meaning, music pieces in the same genre, and books released in the same year. Based on such auxiliary information, we could suppose that parameters in a group have similar roles in a problem and similar values. In this paper, we propose the Higher Order Fused (HOF) regularization that can incorporate smoothness among parameters with group structures as prior knowledge in supervised learning. We define the HOF penalty as the Lovász extension of a submodular higher-order potential function, which encourages parameters in a group to take similar estimated values when used as a regularizer. Moreover, we develop an efficient network flow algorithm for calculating the proximity operator for the regularized problem. We investigate the empirical performance of the proposed algorithm by using synthetic and real-world data.

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings
EditorsAnnalisa Appice, João Gama, Vitor Santos Costa, João Gama, Alípio Jorge, Annalisa Appice, Annalisa Appice, Vitor Santos Costa, Alípio Jorge, Annalisa Appice, Pedro Pereira Rodrigues, Pedro Pereira Rodrigues, João Gama, Vitor Santos Costa, Soares Soares, Pedro Pereira Rodrigues, Soares Soares, Soares Soares, João Gama, Soares Soares, Alípio Jorge, Alípio Jorge, Pedro Pereira Rodrigues, Vitor Santos Costa
PublisherSpringer Verlag
Pages577-593
Number of pages17
ISBN (Print)9783319235271, 9783319235271, 9783319235271, 9783319235271
DOIs
Publication statusPublished - Jan 1 2015
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2015 - Porto, Portugal
Duration: Sep 7 2015Sep 11 2015

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9284
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2015
CountryPortugal
CityPorto
Period9/7/159/11/15

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Higher order fused regularization for supervised learning with grouped parameters'. Together they form a unique fingerprint.

  • Cite this

    Takeuchi, K., Kawahara, Y., & Iwata, T. (2015). Higher order fused regularization for supervised learning with grouped parameters. In A. Appice, J. Gama, V. S. Costa, J. Gama, A. Jorge, A. Appice, A. Appice, V. S. Costa, A. Jorge, A. Appice, P. P. Rodrigues, P. P. Rodrigues, J. Gama, V. S. Costa, S. Soares, P. P. Rodrigues, S. Soares, S. Soares, J. Gama, S. Soares, A. Jorge, A. Jorge, P. P. Rodrigues, ... V. S. Costa (Eds.), Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings (pp. 577-593). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9284). Springer Verlag. https://doi.org/10.1007/978-3-319-23528-8_36