A simple algorithm for predicting nearly as well as the best pruning labeled with the best prediction values of a decision tree

Eiji Takimoto, Ken'ichi Hirai, Akira Maruoka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Given an unpruned decision tree, Helmbold and Schapire gave an on-line prediction algorithm whose performance will not be much worse than the predictions made by the best pruning of the given decision tree. In this paper, based on the observation that finding the best pruning can be efficiently solved by a dynamic programming in the “batch” setting where all the data to be predicted are given in advance, a new on-line prediction algorithm is constructed. Although it is shown that its performance is slightly weaker than Helmbold and Sehapire’s algorithm with respect to the loss bound, it is so simple and general that it could be applied to many on-line optimization problems solved by dynamic programming. We also explore algorithms that are competitive not only with the best pruning but also with the best prediction values. In this setting, a greatly simplified algorithm is given, and it is shown that the algorithm can easily be generalized to the case where, instead of using decision trees, data are classified in some arbitrarily fixed manner.

Original languageEnglish
Title of host publicationAlgorithmic Learning Theory - 8th International Workshop, ALT 1997, Proceedings
EditorsMing Li, Akira Maruoka
PublisherSpringer Verlag
Pages385-400
Number of pages16
ISBN (Print)3540635777, 9783540635772
DOIs
Publication statusPublished - Jan 1 1997
Event8th International Workshop on Algorithmic Learning Theory, ALT 1997 - Sendai, Japan
Duration: Oct 6 1997Oct 8 1997

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume1316
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other8th International Workshop on Algorithmic Learning Theory, ALT 1997
CountryJapan
CitySendai
Period10/6/9710/8/97

Fingerprint

Decision trees
Pruning
Decision tree
Prediction
Dynamic programming
Dynamic Programming
Batch
Optimization Problem

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Takimoto, E., Hirai, K., & Maruoka, A. (1997). A simple algorithm for predicting nearly as well as the best pruning labeled with the best prediction values of a decision tree. In M. Li, & A. Maruoka (Eds.), Algorithmic Learning Theory - 8th International Workshop, ALT 1997, Proceedings (pp. 385-400). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 1316). Springer Verlag. https://doi.org/10.1007/3-540-63577-7_56

A simple algorithm for predicting nearly as well as the best pruning labeled with the best prediction values of a decision tree. / Takimoto, Eiji; Hirai, Ken'ichi; Maruoka, Akira.

Algorithmic Learning Theory - 8th International Workshop, ALT 1997, Proceedings. ed. / Ming Li; Akira Maruoka. Springer Verlag, 1997. p. 385-400 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 1316).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Takimoto, E, Hirai, K & Maruoka, A 1997, A simple algorithm for predicting nearly as well as the best pruning labeled with the best prediction values of a decision tree. in M Li & A Maruoka (eds), Algorithmic Learning Theory - 8th International Workshop, ALT 1997, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 1316, Springer Verlag, pp. 385-400, 8th International Workshop on Algorithmic Learning Theory, ALT 1997, Sendai, Japan, 10/6/97. https://doi.org/10.1007/3-540-63577-7_56
Takimoto E, Hirai K, Maruoka A. A simple algorithm for predicting nearly as well as the best pruning labeled with the best prediction values of a decision tree. In Li M, Maruoka A, editors, Algorithmic Learning Theory - 8th International Workshop, ALT 1997, Proceedings. Springer Verlag. 1997. p. 385-400. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/3-540-63577-7_56
Takimoto, Eiji ; Hirai, Ken'ichi ; Maruoka, Akira. / A simple algorithm for predicting nearly as well as the best pruning labeled with the best prediction values of a decision tree. Algorithmic Learning Theory - 8th International Workshop, ALT 1997, Proceedings. editor / Ming Li ; Akira Maruoka. Springer Verlag, 1997. pp. 385-400 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{80eff1848422442185a5e8975f81929a,
title = "A simple algorithm for predicting nearly as well as the best pruning labeled with the best prediction values of a decision tree",
abstract = "Given an unpruned decision tree, Helmbold and Schapire gave an on-line prediction algorithm whose performance will not be much worse than the predictions made by the best pruning of the given decision tree. In this paper, based on the observation that finding the best pruning can be efficiently solved by a dynamic programming in the “batch” setting where all the data to be predicted are given in advance, a new on-line prediction algorithm is constructed. Although it is shown that its performance is slightly weaker than Helmbold and Sehapire’s algorithm with respect to the loss bound, it is so simple and general that it could be applied to many on-line optimization problems solved by dynamic programming. We also explore algorithms that are competitive not only with the best pruning but also with the best prediction values. In this setting, a greatly simplified algorithm is given, and it is shown that the algorithm can easily be generalized to the case where, instead of using decision trees, data are classified in some arbitrarily fixed manner.",
author = "Eiji Takimoto and Ken'ichi Hirai and Akira Maruoka",
year = "1997",
month = "1",
day = "1",
doi = "10.1007/3-540-63577-7_56",
language = "English",
isbn = "3540635777",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "385--400",
editor = "Ming Li and Akira Maruoka",
booktitle = "Algorithmic Learning Theory - 8th International Workshop, ALT 1997, Proceedings",
address = "Germany",

}

TY - GEN

T1 - A simple algorithm for predicting nearly as well as the best pruning labeled with the best prediction values of a decision tree

AU - Takimoto, Eiji

AU - Hirai, Ken'ichi

AU - Maruoka, Akira

PY - 1997/1/1

Y1 - 1997/1/1

N2 - Given an unpruned decision tree, Helmbold and Schapire gave an on-line prediction algorithm whose performance will not be much worse than the predictions made by the best pruning of the given decision tree. In this paper, based on the observation that finding the best pruning can be efficiently solved by a dynamic programming in the “batch” setting where all the data to be predicted are given in advance, a new on-line prediction algorithm is constructed. Although it is shown that its performance is slightly weaker than Helmbold and Sehapire’s algorithm with respect to the loss bound, it is so simple and general that it could be applied to many on-line optimization problems solved by dynamic programming. We also explore algorithms that are competitive not only with the best pruning but also with the best prediction values. In this setting, a greatly simplified algorithm is given, and it is shown that the algorithm can easily be generalized to the case where, instead of using decision trees, data are classified in some arbitrarily fixed manner.

AB - Given an unpruned decision tree, Helmbold and Schapire gave an on-line prediction algorithm whose performance will not be much worse than the predictions made by the best pruning of the given decision tree. In this paper, based on the observation that finding the best pruning can be efficiently solved by a dynamic programming in the “batch” setting where all the data to be predicted are given in advance, a new on-line prediction algorithm is constructed. Although it is shown that its performance is slightly weaker than Helmbold and Sehapire’s algorithm with respect to the loss bound, it is so simple and general that it could be applied to many on-line optimization problems solved by dynamic programming. We also explore algorithms that are competitive not only with the best pruning but also with the best prediction values. In this setting, a greatly simplified algorithm is given, and it is shown that the algorithm can easily be generalized to the case where, instead of using decision trees, data are classified in some arbitrarily fixed manner.

UR - http://www.scopus.com/inward/record.url?scp=84961377259&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84961377259&partnerID=8YFLogxK

U2 - 10.1007/3-540-63577-7_56

DO - 10.1007/3-540-63577-7_56

M3 - Conference contribution

AN - SCOPUS:84961377259

SN - 3540635777

SN - 9783540635772

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 385

EP - 400

BT - Algorithmic Learning Theory - 8th International Workshop, ALT 1997, Proceedings

A2 - Li, Ming

A2 - Maruoka, Akira

PB - Springer Verlag

ER -