Top-down decision tree boosting and its applications

Eiji Takimoto, Akira Maruoka

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

Top-down algorithms such as C4.5 and CART for constructing decision trees are known to perform boosting, with the procedure of choosing classification rules at internal nodes regarded as the base learner. In this work, by introducing a notion of pseudo-entropy functions for measuring the loss of hypotheses, we give a new insight into this boosting scheme from an information-theoretic viewpoint: Whenever the base learner produces hypotheses with non-zero mutual information, the top-down algorithm reduces the conditional entropy (uncertainty) about the target function as the tree grows. Although its theoretical guarantee on its performance is worse than other popular boosting algorithms such as AdaBoost, the top-down algorithms can naturally treat multiclass classification problems. Furthermore we propose a base learner LIN that produces linear classification functions and carry out some experiments to examine the performance of the top-down algorithm with LIN as the base learner. The results show that the algorithm can sometimes perform as well as or better than AdaBoost.

Original languageEnglish
Title of host publicationProgress in Discovery Science
PublisherSpringer Verlag
Pages327-337
Number of pages11
ISBN (Print)3540433384, 9783540433385
DOIs
Publication statusPublished - 2002
Externally publishedYes

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume2281
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Top-down decision tree boosting and its applications'. Together they form a unique fingerprint.

  • Cite this

    Takimoto, E., & Maruoka, A. (2002). Top-down decision tree boosting and its applications. In Progress in Discovery Science (pp. 327-337). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2281). Springer Verlag. https://doi.org/10.1007/3-540-45884-0_23