Boosting versus covering

kohei hatano, Manfred K. Warmuth

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We investigate improvements of AdaBoost that can exploit the fact that the weak hypotheses are one-sided, i.e. either all its positive (or negative) predictions are correct. In particular, for any set of m labeled examples consistent with a disjunction of literals (which are one-sided in this case), AdaBoost constructs a consistent hypothesis by using O(2 logm) iterations. On the other hand, a greedy set covering algorithm finds a consistent hypothesis of size O( logm). Our primary question is whether there is a simple boosting algorithm that performs as well as the greedy set covering. We first show that InfoBoost, a modification of AdaBoost proposed by Aslam for a different purpose, does perform as well as the greedy set covering algorithm. We then show that AdaBoost requires Ω(2 logm) iterations for learning -literal disjunctions. We achieve this with an adversary construction and as well as in simple experiments based on artificial data. Further we give a variant called SemiBoost that can handle the degenerate case when the given examples all have the same label. We conclude by showing that SemiBoost can be used to produce small conjunctions as well.

Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003
PublisherNeural information processing systems foundation
ISBN (Print)0262201526, 9780262201520
Publication statusPublished - 2004
Externally publishedYes
Event17th Annual Conference on Neural Information Processing Systems, NIPS 2003 - Vancouver, BC, Canada
Duration: Dec 8 2003Dec 13 2003

Other

Other17th Annual Conference on Neural Information Processing Systems, NIPS 2003
CountryCanada
CityVancouver, BC
Period12/8/0312/13/03

Fingerprint

Adaptive boosting
Labels
Experiments

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Cite this

hatano, K., & Warmuth, M. K. (2004). Boosting versus covering. In Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003 Neural information processing systems foundation.

Boosting versus covering. / hatano, kohei; Warmuth, Manfred K.

Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003. Neural information processing systems foundation, 2004.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

hatano, K & Warmuth, MK 2004, Boosting versus covering. in Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003. Neural information processing systems foundation, 17th Annual Conference on Neural Information Processing Systems, NIPS 2003, Vancouver, BC, Canada, 12/8/03.
hatano K, Warmuth MK. Boosting versus covering. In Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003. Neural information processing systems foundation. 2004
hatano, kohei ; Warmuth, Manfred K. / Boosting versus covering. Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003. Neural information processing systems foundation, 2004.
@inproceedings{718f8a2ca9e14157b99dab0dd5944557,
title = "Boosting versus covering",
abstract = "We investigate improvements of AdaBoost that can exploit the fact that the weak hypotheses are one-sided, i.e. either all its positive (or negative) predictions are correct. In particular, for any set of m labeled examples consistent with a disjunction of literals (which are one-sided in this case), AdaBoost constructs a consistent hypothesis by using O(2 logm) iterations. On the other hand, a greedy set covering algorithm finds a consistent hypothesis of size O( logm). Our primary question is whether there is a simple boosting algorithm that performs as well as the greedy set covering. We first show that InfoBoost, a modification of AdaBoost proposed by Aslam for a different purpose, does perform as well as the greedy set covering algorithm. We then show that AdaBoost requires Ω(2 logm) iterations for learning -literal disjunctions. We achieve this with an adversary construction and as well as in simple experiments based on artificial data. Further we give a variant called SemiBoost that can handle the degenerate case when the given examples all have the same label. We conclude by showing that SemiBoost can be used to produce small conjunctions as well.",
author = "kohei hatano and Warmuth, {Manfred K.}",
year = "2004",
language = "English",
isbn = "0262201526",
booktitle = "Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003",
publisher = "Neural information processing systems foundation",

}

TY - GEN

T1 - Boosting versus covering

AU - hatano, kohei

AU - Warmuth, Manfred K.

PY - 2004

Y1 - 2004

N2 - We investigate improvements of AdaBoost that can exploit the fact that the weak hypotheses are one-sided, i.e. either all its positive (or negative) predictions are correct. In particular, for any set of m labeled examples consistent with a disjunction of literals (which are one-sided in this case), AdaBoost constructs a consistent hypothesis by using O(2 logm) iterations. On the other hand, a greedy set covering algorithm finds a consistent hypothesis of size O( logm). Our primary question is whether there is a simple boosting algorithm that performs as well as the greedy set covering. We first show that InfoBoost, a modification of AdaBoost proposed by Aslam for a different purpose, does perform as well as the greedy set covering algorithm. We then show that AdaBoost requires Ω(2 logm) iterations for learning -literal disjunctions. We achieve this with an adversary construction and as well as in simple experiments based on artificial data. Further we give a variant called SemiBoost that can handle the degenerate case when the given examples all have the same label. We conclude by showing that SemiBoost can be used to produce small conjunctions as well.

AB - We investigate improvements of AdaBoost that can exploit the fact that the weak hypotheses are one-sided, i.e. either all its positive (or negative) predictions are correct. In particular, for any set of m labeled examples consistent with a disjunction of literals (which are one-sided in this case), AdaBoost constructs a consistent hypothesis by using O(2 logm) iterations. On the other hand, a greedy set covering algorithm finds a consistent hypothesis of size O( logm). Our primary question is whether there is a simple boosting algorithm that performs as well as the greedy set covering. We first show that InfoBoost, a modification of AdaBoost proposed by Aslam for a different purpose, does perform as well as the greedy set covering algorithm. We then show that AdaBoost requires Ω(2 logm) iterations for learning -literal disjunctions. We achieve this with an adversary construction and as well as in simple experiments based on artificial data. Further we give a variant called SemiBoost that can handle the degenerate case when the given examples all have the same label. We conclude by showing that SemiBoost can be used to produce small conjunctions as well.

UR - http://www.scopus.com/inward/record.url?scp=84860966920&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84860966920&partnerID=8YFLogxK

M3 - Conference contribution

SN - 0262201526

SN - 9780262201520

BT - Advances in Neural Information Processing Systems 16 - Proceedings of the 2003 Conference, NIPS 2003

PB - Neural information processing systems foundation

ER -