Submodularity cuts and applications

Yoshinobu Kawahara, Kiyohito Nagano, Koji Tsuda, Jeff A. Bilmes

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Citations (Scopus)

Abstract

Several key problems in machine learning, such as feature selection and active learning, can be formulated as submodular set function maximization. We present herein a novel algorithm for maximizing a submodular set function under a cardinality constraint - the algorithm is based on a cutting-plane method and is implemented as an iterative small-scale binary-integer linear programming procedure. It is well known that this problem is NP-hard, and the approximation factor achieved by the greedy algorithm is the theoretical limit for polynomial time. As for (non-polynomial time) exact algorithms that perform reasonably in practice, there has been very little in the literature although the problem is quite important for many applications. Our algorithm is guaranteed to find the exact solution finitely many iterations, and it converges fast in practice due to the efficiency of the cutting-plane mechanism. Moreover, we also provide a method that produces successively decreasing upper-bounds of the optimal solution, while our algorithm provides successively increasing lower-bounds. Thus, the accuracy of the current solution can be estimated at any point, and the algorithm can be stopped early once a desired degree of tolerance is met. We evaluate our algorithm on sensor placement and feature selection applications showing good performance.

Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference
Pages916-924
Number of pages9
Publication statusPublished - Dec 1 2009
Externally publishedYes
Event23rd Annual Conference on Neural Information Processing Systems, NIPS 2009 - Vancouver, BC, Canada
Duration: Dec 7 2009Dec 10 2009

Publication series

NameAdvances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference

Conference

Conference23rd Annual Conference on Neural Information Processing Systems, NIPS 2009
CountryCanada
CityVancouver, BC
Period12/7/0912/10/09

Fingerprint

Feature extraction
Set theory
Linear programming
Learning systems
Computational complexity
Polynomials
Sensors
Problem-Based Learning

All Science Journal Classification (ASJC) codes

  • Information Systems

Cite this

Kawahara, Y., Nagano, K., Tsuda, K., & Bilmes, J. A. (2009). Submodularity cuts and applications. In Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference (pp. 916-924). (Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference).

Submodularity cuts and applications. / Kawahara, Yoshinobu; Nagano, Kiyohito; Tsuda, Koji; Bilmes, Jeff A.

Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference. 2009. p. 916-924 (Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kawahara, Y, Nagano, K, Tsuda, K & Bilmes, JA 2009, Submodularity cuts and applications. in Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference. Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference, pp. 916-924, 23rd Annual Conference on Neural Information Processing Systems, NIPS 2009, Vancouver, BC, Canada, 12/7/09.
Kawahara Y, Nagano K, Tsuda K, Bilmes JA. Submodularity cuts and applications. In Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference. 2009. p. 916-924. (Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference).
Kawahara, Yoshinobu ; Nagano, Kiyohito ; Tsuda, Koji ; Bilmes, Jeff A. / Submodularity cuts and applications. Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference. 2009. pp. 916-924 (Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference).
@inproceedings{6fa4436bbde34f1ea3109094299da5b0,
title = "Submodularity cuts and applications",
abstract = "Several key problems in machine learning, such as feature selection and active learning, can be formulated as submodular set function maximization. We present herein a novel algorithm for maximizing a submodular set function under a cardinality constraint - the algorithm is based on a cutting-plane method and is implemented as an iterative small-scale binary-integer linear programming procedure. It is well known that this problem is NP-hard, and the approximation factor achieved by the greedy algorithm is the theoretical limit for polynomial time. As for (non-polynomial time) exact algorithms that perform reasonably in practice, there has been very little in the literature although the problem is quite important for many applications. Our algorithm is guaranteed to find the exact solution finitely many iterations, and it converges fast in practice due to the efficiency of the cutting-plane mechanism. Moreover, we also provide a method that produces successively decreasing upper-bounds of the optimal solution, while our algorithm provides successively increasing lower-bounds. Thus, the accuracy of the current solution can be estimated at any point, and the algorithm can be stopped early once a desired degree of tolerance is met. We evaluate our algorithm on sensor placement and feature selection applications showing good performance.",
author = "Yoshinobu Kawahara and Kiyohito Nagano and Koji Tsuda and Bilmes, {Jeff A.}",
year = "2009",
month = "12",
day = "1",
language = "English",
isbn = "9781615679119",
series = "Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference",
pages = "916--924",
booktitle = "Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference",

}

TY - GEN

T1 - Submodularity cuts and applications

AU - Kawahara, Yoshinobu

AU - Nagano, Kiyohito

AU - Tsuda, Koji

AU - Bilmes, Jeff A.

PY - 2009/12/1

Y1 - 2009/12/1

N2 - Several key problems in machine learning, such as feature selection and active learning, can be formulated as submodular set function maximization. We present herein a novel algorithm for maximizing a submodular set function under a cardinality constraint - the algorithm is based on a cutting-plane method and is implemented as an iterative small-scale binary-integer linear programming procedure. It is well known that this problem is NP-hard, and the approximation factor achieved by the greedy algorithm is the theoretical limit for polynomial time. As for (non-polynomial time) exact algorithms that perform reasonably in practice, there has been very little in the literature although the problem is quite important for many applications. Our algorithm is guaranteed to find the exact solution finitely many iterations, and it converges fast in practice due to the efficiency of the cutting-plane mechanism. Moreover, we also provide a method that produces successively decreasing upper-bounds of the optimal solution, while our algorithm provides successively increasing lower-bounds. Thus, the accuracy of the current solution can be estimated at any point, and the algorithm can be stopped early once a desired degree of tolerance is met. We evaluate our algorithm on sensor placement and feature selection applications showing good performance.

AB - Several key problems in machine learning, such as feature selection and active learning, can be formulated as submodular set function maximization. We present herein a novel algorithm for maximizing a submodular set function under a cardinality constraint - the algorithm is based on a cutting-plane method and is implemented as an iterative small-scale binary-integer linear programming procedure. It is well known that this problem is NP-hard, and the approximation factor achieved by the greedy algorithm is the theoretical limit for polynomial time. As for (non-polynomial time) exact algorithms that perform reasonably in practice, there has been very little in the literature although the problem is quite important for many applications. Our algorithm is guaranteed to find the exact solution finitely many iterations, and it converges fast in practice due to the efficiency of the cutting-plane mechanism. Moreover, we also provide a method that produces successively decreasing upper-bounds of the optimal solution, while our algorithm provides successively increasing lower-bounds. Thus, the accuracy of the current solution can be estimated at any point, and the algorithm can be stopped early once a desired degree of tolerance is met. We evaluate our algorithm on sensor placement and feature selection applications showing good performance.

UR - http://www.scopus.com/inward/record.url?scp=84858740625&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84858740625&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781615679119

T3 - Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference

SP - 916

EP - 924

BT - Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference

ER -