Higher order fused regularization for supervised learning with grouped parameters

Koh Takeuchi, Yoshinobu Kawahara, Tomoharu Iwata

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We often encounter situations in supervised learning where there exist possibly groups that consist of more than two parameters. For example, we might work on parameters that correspond to words expressing the same meaning, music pieces in the same genre, and books released in the same year. Based on such auxiliary information, we could suppose that parameters in a group have similar roles in a problem and similar values. In this paper, we propose the Higher Order Fused (HOF) regularization that can incorporate smoothness among parameters with group structures as prior knowledge in supervised learning. We define the HOF penalty as the Lovász extension of a submodular higher-order potential function, which encourages parameters in a group to take similar estimated values when used as a regularizer. Moreover, we develop an efficient network flow algorithm for calculating the proximity operator for the regularized problem. We investigate the empirical performance of the proposed algorithm by using synthetic and real-world data.

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings
EditorsAnnalisa Appice, João Gama, Vitor Santos Costa, João Gama, Alípio Jorge, Annalisa Appice, Annalisa Appice, Vitor Santos Costa, Alípio Jorge, Annalisa Appice, Pedro Pereira Rodrigues, Pedro Pereira Rodrigues, João Gama, Vitor Santos Costa, Soares Soares, Pedro Pereira Rodrigues, Soares Soares, Soares Soares, João Gama, Soares Soares, Alípio Jorge, Alípio Jorge, Pedro Pereira Rodrigues, Vitor Santos Costa
PublisherSpringer Verlag
Pages577-593
Number of pages17
ISBN (Print)9783319235271, 9783319235271, 9783319235271, 9783319235271
DOIs
Publication statusPublished - Jan 1 2015
Externally publishedYes
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2015 - Porto, Portugal
Duration: Sep 7 2015Sep 11 2015

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9284
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2015
CountryPortugal
CityPorto
Period9/7/159/11/15

Fingerprint

Supervised learning
Supervised Learning
Regularization
Higher Order
Mathematical operators
Auxiliary Information
Network Flow
Potential Function
Music
Prior Knowledge
Proximity
Penalty
Two Parameters
Smoothness
Operator

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Takeuchi, K., Kawahara, Y., & Iwata, T. (2015). Higher order fused regularization for supervised learning with grouped parameters. In A. Appice, J. Gama, V. S. Costa, J. Gama, A. Jorge, A. Appice, A. Appice, V. S. Costa, A. Jorge, A. Appice, P. P. Rodrigues, P. P. Rodrigues, J. Gama, V. S. Costa, S. Soares, P. P. Rodrigues, S. Soares, S. Soares, J. Gama, S. Soares, A. Jorge, A. Jorge, P. P. Rodrigues, ... V. S. Costa (Eds.), Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings (pp. 577-593). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9284). Springer Verlag. https://doi.org/10.1007/978-3-319-23528-8_36

Higher order fused regularization for supervised learning with grouped parameters. / Takeuchi, Koh; Kawahara, Yoshinobu; Iwata, Tomoharu.

Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings. ed. / Annalisa Appice; João Gama; Vitor Santos Costa; João Gama; Alípio Jorge; Annalisa Appice; Annalisa Appice; Vitor Santos Costa; Alípio Jorge; Annalisa Appice; Pedro Pereira Rodrigues; Pedro Pereira Rodrigues; João Gama; Vitor Santos Costa; Soares Soares; Pedro Pereira Rodrigues; Soares Soares; Soares Soares; João Gama; Soares Soares; Alípio Jorge; Alípio Jorge; Pedro Pereira Rodrigues; Vitor Santos Costa. Springer Verlag, 2015. p. 577-593 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9284).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Takeuchi, K, Kawahara, Y & Iwata, T 2015, Higher order fused regularization for supervised learning with grouped parameters. in A Appice, J Gama, VS Costa, J Gama, A Jorge, A Appice, A Appice, VS Costa, A Jorge, A Appice, PP Rodrigues, PP Rodrigues, J Gama, VS Costa, S Soares, PP Rodrigues, S Soares, S Soares, J Gama, S Soares, A Jorge, A Jorge, PP Rodrigues & VS Costa (eds), Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9284, Springer Verlag, pp. 577-593, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2015, Porto, Portugal, 9/7/15. https://doi.org/10.1007/978-3-319-23528-8_36
Takeuchi K, Kawahara Y, Iwata T. Higher order fused regularization for supervised learning with grouped parameters. In Appice A, Gama J, Costa VS, Gama J, Jorge A, Appice A, Appice A, Costa VS, Jorge A, Appice A, Rodrigues PP, Rodrigues PP, Gama J, Costa VS, Soares S, Rodrigues PP, Soares S, Soares S, Gama J, Soares S, Jorge A, Jorge A, Rodrigues PP, Costa VS, editors, Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings. Springer Verlag. 2015. p. 577-593. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-23528-8_36
Takeuchi, Koh ; Kawahara, Yoshinobu ; Iwata, Tomoharu. / Higher order fused regularization for supervised learning with grouped parameters. Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings. editor / Annalisa Appice ; João Gama ; Vitor Santos Costa ; João Gama ; Alípio Jorge ; Annalisa Appice ; Annalisa Appice ; Vitor Santos Costa ; Alípio Jorge ; Annalisa Appice ; Pedro Pereira Rodrigues ; Pedro Pereira Rodrigues ; João Gama ; Vitor Santos Costa ; Soares Soares ; Pedro Pereira Rodrigues ; Soares Soares ; Soares Soares ; João Gama ; Soares Soares ; Alípio Jorge ; Alípio Jorge ; Pedro Pereira Rodrigues ; Vitor Santos Costa. Springer Verlag, 2015. pp. 577-593 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{03797c7013fc488f8feca5a1fcc8a01b,
title = "Higher order fused regularization for supervised learning with grouped parameters",
abstract = "We often encounter situations in supervised learning where there exist possibly groups that consist of more than two parameters. For example, we might work on parameters that correspond to words expressing the same meaning, music pieces in the same genre, and books released in the same year. Based on such auxiliary information, we could suppose that parameters in a group have similar roles in a problem and similar values. In this paper, we propose the Higher Order Fused (HOF) regularization that can incorporate smoothness among parameters with group structures as prior knowledge in supervised learning. We define the HOF penalty as the Lov{\'a}sz extension of a submodular higher-order potential function, which encourages parameters in a group to take similar estimated values when used as a regularizer. Moreover, we develop an efficient network flow algorithm for calculating the proximity operator for the regularized problem. We investigate the empirical performance of the proposed algorithm by using synthetic and real-world data.",
author = "Koh Takeuchi and Yoshinobu Kawahara and Tomoharu Iwata",
year = "2015",
month = "1",
day = "1",
doi = "10.1007/978-3-319-23528-8_36",
language = "English",
isbn = "9783319235271",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "577--593",
editor = "Annalisa Appice and Jo{\~a}o Gama and Costa, {Vitor Santos} and Jo{\~a}o Gama and Al{\'i}pio Jorge and Annalisa Appice and Annalisa Appice and Costa, {Vitor Santos} and Al{\'i}pio Jorge and Annalisa Appice and Rodrigues, {Pedro Pereira} and Rodrigues, {Pedro Pereira} and Jo{\~a}o Gama and Costa, {Vitor Santos} and Soares Soares and Rodrigues, {Pedro Pereira} and Soares Soares and Soares Soares and Jo{\~a}o Gama and Soares Soares and Al{\'i}pio Jorge and Al{\'i}pio Jorge and Rodrigues, {Pedro Pereira} and Costa, {Vitor Santos}",
booktitle = "Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings",
address = "Germany",

}

TY - GEN

T1 - Higher order fused regularization for supervised learning with grouped parameters

AU - Takeuchi, Koh

AU - Kawahara, Yoshinobu

AU - Iwata, Tomoharu

PY - 2015/1/1

Y1 - 2015/1/1

N2 - We often encounter situations in supervised learning where there exist possibly groups that consist of more than two parameters. For example, we might work on parameters that correspond to words expressing the same meaning, music pieces in the same genre, and books released in the same year. Based on such auxiliary information, we could suppose that parameters in a group have similar roles in a problem and similar values. In this paper, we propose the Higher Order Fused (HOF) regularization that can incorporate smoothness among parameters with group structures as prior knowledge in supervised learning. We define the HOF penalty as the Lovász extension of a submodular higher-order potential function, which encourages parameters in a group to take similar estimated values when used as a regularizer. Moreover, we develop an efficient network flow algorithm for calculating the proximity operator for the regularized problem. We investigate the empirical performance of the proposed algorithm by using synthetic and real-world data.

AB - We often encounter situations in supervised learning where there exist possibly groups that consist of more than two parameters. For example, we might work on parameters that correspond to words expressing the same meaning, music pieces in the same genre, and books released in the same year. Based on such auxiliary information, we could suppose that parameters in a group have similar roles in a problem and similar values. In this paper, we propose the Higher Order Fused (HOF) regularization that can incorporate smoothness among parameters with group structures as prior knowledge in supervised learning. We define the HOF penalty as the Lovász extension of a submodular higher-order potential function, which encourages parameters in a group to take similar estimated values when used as a regularizer. Moreover, we develop an efficient network flow algorithm for calculating the proximity operator for the regularized problem. We investigate the empirical performance of the proposed algorithm by using synthetic and real-world data.

UR - http://www.scopus.com/inward/record.url?scp=84984650204&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84984650204&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-23528-8_36

DO - 10.1007/978-3-319-23528-8_36

M3 - Conference contribution

SN - 9783319235271

SN - 9783319235271

SN - 9783319235271

SN - 9783319235271

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 577

EP - 593

BT - Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Proceedings

A2 - Appice, Annalisa

A2 - Gama, João

A2 - Costa, Vitor Santos

A2 - Gama, João

A2 - Jorge, Alípio

A2 - Appice, Annalisa

A2 - Appice, Annalisa

A2 - Costa, Vitor Santos

A2 - Jorge, Alípio

A2 - Appice, Annalisa

A2 - Rodrigues, Pedro Pereira

A2 - Rodrigues, Pedro Pereira

A2 - Gama, João

A2 - Costa, Vitor Santos

A2 - Soares, Soares

A2 - Rodrigues, Pedro Pereira

A2 - Soares, Soares

A2 - Soares, Soares

A2 - Gama, João

A2 - Soares, Soares

A2 - Jorge, Alípio

A2 - Jorge, Alípio

A2 - Rodrigues, Pedro Pereira

A2 - Costa, Vitor Santos

PB - Springer Verlag

ER -