On the boosting algorithm for multiclass functions based on information-theoretic criterion for approximation

Eiji Takimoto, Akira Maruoka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

We consider the boosting technique that can be directly applied to the classification problem for multiclass functions. Although many boosting algorithms have been proposed so far, all of them are essentially developed for binary classification problems, and in order to handle multiclass classification problems, they need the problems reduced somehow to binary ones. In order to avoid such reductions, we introduce a notion of the pseudo-entropy function G that gives an information-theoretic criterion, called the conditional G-entropy, for measuring the loss of hypotheses. The conditional G-entropy turns out to be useful for defining the weakness of hypotheses that approximate, in some way, to a multiclass function in general, so that we can consider the boosting problem without reduction. We show that the top-down decision tree learning algorithm using G as its splitting criterion is an efficient boosting algorithm based on the conditional G-entropy. Namely, the algorithm intends to minimize the conditional G-entropy, rather than the classification error. In the binary case, our algorithm is identical to the error-based boosting algorithm proposed by Kearns and Mansour, and our analysis gives a simpler proof of their results.

Original languageEnglish
Title of host publicationDiscovery Science - 1st International Conference, DS 1998, Proceedings
EditorsSetsuo Arikawa, Hiroshi Motoda
PublisherSpringer Verlag
Pages256-267
Number of pages12
ISBN (Print)3540653902, 9783540653905
Publication statusPublished - Jan 1 1998
Externally publishedYes
Event1st International Conference on Discovery Science, DS 1998 - Fukuoka, Japan
Duration: Dec 14 1998Dec 16 1998

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume1532
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other1st International Conference on Discovery Science, DS 1998
CountryJapan
CityFukuoka
Period12/14/9812/16/98

Fingerprint

Multi-class
Boosting
Entropy
Classification Problems
Approximation
Binary
Multi-class Classification
Entropy Function
Binary Classification
Tree Algorithms
Decision trees
Decision tree
Learning algorithms
Learning Algorithm
Minimise

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Takimoto, E., & Maruoka, A. (1998). On the boosting algorithm for multiclass functions based on information-theoretic criterion for approximation. In S. Arikawa, & H. Motoda (Eds.), Discovery Science - 1st International Conference, DS 1998, Proceedings (pp. 256-267). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 1532). Springer Verlag.

On the boosting algorithm for multiclass functions based on information-theoretic criterion for approximation. / Takimoto, Eiji; Maruoka, Akira.

Discovery Science - 1st International Conference, DS 1998, Proceedings. ed. / Setsuo Arikawa; Hiroshi Motoda. Springer Verlag, 1998. p. 256-267 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 1532).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Takimoto, E & Maruoka, A 1998, On the boosting algorithm for multiclass functions based on information-theoretic criterion for approximation. in S Arikawa & H Motoda (eds), Discovery Science - 1st International Conference, DS 1998, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 1532, Springer Verlag, pp. 256-267, 1st International Conference on Discovery Science, DS 1998, Fukuoka, Japan, 12/14/98.
Takimoto E, Maruoka A. On the boosting algorithm for multiclass functions based on information-theoretic criterion for approximation. In Arikawa S, Motoda H, editors, Discovery Science - 1st International Conference, DS 1998, Proceedings. Springer Verlag. 1998. p. 256-267. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Takimoto, Eiji ; Maruoka, Akira. / On the boosting algorithm for multiclass functions based on information-theoretic criterion for approximation. Discovery Science - 1st International Conference, DS 1998, Proceedings. editor / Setsuo Arikawa ; Hiroshi Motoda. Springer Verlag, 1998. pp. 256-267 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{405cd71c3417456aa18e057868c29011,
title = "On the boosting algorithm for multiclass functions based on information-theoretic criterion for approximation",
abstract = "We consider the boosting technique that can be directly applied to the classification problem for multiclass functions. Although many boosting algorithms have been proposed so far, all of them are essentially developed for binary classification problems, and in order to handle multiclass classification problems, they need the problems reduced somehow to binary ones. In order to avoid such reductions, we introduce a notion of the pseudo-entropy function G that gives an information-theoretic criterion, called the conditional G-entropy, for measuring the loss of hypotheses. The conditional G-entropy turns out to be useful for defining the weakness of hypotheses that approximate, in some way, to a multiclass function in general, so that we can consider the boosting problem without reduction. We show that the top-down decision tree learning algorithm using G as its splitting criterion is an efficient boosting algorithm based on the conditional G-entropy. Namely, the algorithm intends to minimize the conditional G-entropy, rather than the classification error. In the binary case, our algorithm is identical to the error-based boosting algorithm proposed by Kearns and Mansour, and our analysis gives a simpler proof of their results.",
author = "Eiji Takimoto and Akira Maruoka",
year = "1998",
month = "1",
day = "1",
language = "English",
isbn = "3540653902",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "256--267",
editor = "Setsuo Arikawa and Hiroshi Motoda",
booktitle = "Discovery Science - 1st International Conference, DS 1998, Proceedings",
address = "Germany",

}

TY - GEN

T1 - On the boosting algorithm for multiclass functions based on information-theoretic criterion for approximation

AU - Takimoto, Eiji

AU - Maruoka, Akira

PY - 1998/1/1

Y1 - 1998/1/1

N2 - We consider the boosting technique that can be directly applied to the classification problem for multiclass functions. Although many boosting algorithms have been proposed so far, all of them are essentially developed for binary classification problems, and in order to handle multiclass classification problems, they need the problems reduced somehow to binary ones. In order to avoid such reductions, we introduce a notion of the pseudo-entropy function G that gives an information-theoretic criterion, called the conditional G-entropy, for measuring the loss of hypotheses. The conditional G-entropy turns out to be useful for defining the weakness of hypotheses that approximate, in some way, to a multiclass function in general, so that we can consider the boosting problem without reduction. We show that the top-down decision tree learning algorithm using G as its splitting criterion is an efficient boosting algorithm based on the conditional G-entropy. Namely, the algorithm intends to minimize the conditional G-entropy, rather than the classification error. In the binary case, our algorithm is identical to the error-based boosting algorithm proposed by Kearns and Mansour, and our analysis gives a simpler proof of their results.

AB - We consider the boosting technique that can be directly applied to the classification problem for multiclass functions. Although many boosting algorithms have been proposed so far, all of them are essentially developed for binary classification problems, and in order to handle multiclass classification problems, they need the problems reduced somehow to binary ones. In order to avoid such reductions, we introduce a notion of the pseudo-entropy function G that gives an information-theoretic criterion, called the conditional G-entropy, for measuring the loss of hypotheses. The conditional G-entropy turns out to be useful for defining the weakness of hypotheses that approximate, in some way, to a multiclass function in general, so that we can consider the boosting problem without reduction. We show that the top-down decision tree learning algorithm using G as its splitting criterion is an efficient boosting algorithm based on the conditional G-entropy. Namely, the algorithm intends to minimize the conditional G-entropy, rather than the classification error. In the binary case, our algorithm is identical to the error-based boosting algorithm proposed by Kearns and Mansour, and our analysis gives a simpler proof of their results.

UR - http://www.scopus.com/inward/record.url?scp=84949210605&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84949210605&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84949210605

SN - 3540653902

SN - 9783540653905

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 256

EP - 267

BT - Discovery Science - 1st International Conference, DS 1998, Proceedings

A2 - Arikawa, Setsuo

A2 - Motoda, Hiroshi

PB - Springer Verlag

ER -