Predicting nearly as well as the best pruning of a decision tree through dynamic programming scheme

Eiji Takimoto, Akira Maruoka, Volodya Vovk

Research output: Contribution to journalConference article

11 Citations (Scopus)

Abstract

Helmbold and Schapire gave an on-line prediction algorithm that, when given an unpruned decision tree, produces predictions not much worse than the predictions made by the best pruning of the given decision tree. In this paper, we give two new on-line algorithms. The first algorithm is based on the observation that finding the best pruning can be efficiently solved by a dynamic programming in the "batch" setting where all the data to be predicted are given in advance. This algorithm works well for a wide class of loss functions, whereas the one given by Helmbold and Schapire is only described for the absolute loss function. Moreover, the algorithm given in this paper is so simple and general that it could be applied to many other on-line optimization problems solved by dynamic programming. We also explore the second algorithm that is competitive not only with the best pruning but also with the best prediction values which are associated with nodes in the decision tree. In this setting, a greatly simplified algorithm is given for the absolute loss function. It can be easily generalized to the case where, instead of using decision trees, data are classified in some arbitrarily fixed manner.

Original languageEnglish
Pages (from-to)179-209
Number of pages31
JournalTheoretical Computer Science
Volume261
Issue number1
DOIs
Publication statusPublished - Aug 1 2001
Externally publishedYes
Event8th International Workshop on Algorithmic Learning Theory - Sendai, Japan
Duration: Oct 6 1997Oct 8 1997

Fingerprint

Decision trees
Pruning
Dynamic programming
Decision tree
Dynamic Programming
Loss Function
Prediction
Batch
Optimization Problem
Vertex of a graph

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Predicting nearly as well as the best pruning of a decision tree through dynamic programming scheme. / Takimoto, Eiji; Maruoka, Akira; Vovk, Volodya.

In: Theoretical Computer Science, Vol. 261, No. 1, 01.08.2001, p. 179-209.

Research output: Contribution to journalConference article

@article{cd56665adcef4d4095584999489c7035,
title = "Predicting nearly as well as the best pruning of a decision tree through dynamic programming scheme",
abstract = "Helmbold and Schapire gave an on-line prediction algorithm that, when given an unpruned decision tree, produces predictions not much worse than the predictions made by the best pruning of the given decision tree. In this paper, we give two new on-line algorithms. The first algorithm is based on the observation that finding the best pruning can be efficiently solved by a dynamic programming in the {"}batch{"} setting where all the data to be predicted are given in advance. This algorithm works well for a wide class of loss functions, whereas the one given by Helmbold and Schapire is only described for the absolute loss function. Moreover, the algorithm given in this paper is so simple and general that it could be applied to many other on-line optimization problems solved by dynamic programming. We also explore the second algorithm that is competitive not only with the best pruning but also with the best prediction values which are associated with nodes in the decision tree. In this setting, a greatly simplified algorithm is given for the absolute loss function. It can be easily generalized to the case where, instead of using decision trees, data are classified in some arbitrarily fixed manner.",
author = "Eiji Takimoto and Akira Maruoka and Volodya Vovk",
year = "2001",
month = "8",
day = "1",
doi = "10.1016/S0304-3975(00)00138-9",
language = "English",
volume = "261",
pages = "179--209",
journal = "Theoretical Computer Science",
issn = "0304-3975",
publisher = "Elsevier",
number = "1",

}

TY - JOUR

T1 - Predicting nearly as well as the best pruning of a decision tree through dynamic programming scheme

AU - Takimoto, Eiji

AU - Maruoka, Akira

AU - Vovk, Volodya

PY - 2001/8/1

Y1 - 2001/8/1

N2 - Helmbold and Schapire gave an on-line prediction algorithm that, when given an unpruned decision tree, produces predictions not much worse than the predictions made by the best pruning of the given decision tree. In this paper, we give two new on-line algorithms. The first algorithm is based on the observation that finding the best pruning can be efficiently solved by a dynamic programming in the "batch" setting where all the data to be predicted are given in advance. This algorithm works well for a wide class of loss functions, whereas the one given by Helmbold and Schapire is only described for the absolute loss function. Moreover, the algorithm given in this paper is so simple and general that it could be applied to many other on-line optimization problems solved by dynamic programming. We also explore the second algorithm that is competitive not only with the best pruning but also with the best prediction values which are associated with nodes in the decision tree. In this setting, a greatly simplified algorithm is given for the absolute loss function. It can be easily generalized to the case where, instead of using decision trees, data are classified in some arbitrarily fixed manner.

AB - Helmbold and Schapire gave an on-line prediction algorithm that, when given an unpruned decision tree, produces predictions not much worse than the predictions made by the best pruning of the given decision tree. In this paper, we give two new on-line algorithms. The first algorithm is based on the observation that finding the best pruning can be efficiently solved by a dynamic programming in the "batch" setting where all the data to be predicted are given in advance. This algorithm works well for a wide class of loss functions, whereas the one given by Helmbold and Schapire is only described for the absolute loss function. Moreover, the algorithm given in this paper is so simple and general that it could be applied to many other on-line optimization problems solved by dynamic programming. We also explore the second algorithm that is competitive not only with the best pruning but also with the best prediction values which are associated with nodes in the decision tree. In this setting, a greatly simplified algorithm is given for the absolute loss function. It can be easily generalized to the case where, instead of using decision trees, data are classified in some arbitrarily fixed manner.

UR - http://www.scopus.com/inward/record.url?scp=0034925309&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0034925309&partnerID=8YFLogxK

U2 - 10.1016/S0304-3975(00)00138-9

DO - 10.1016/S0304-3975(00)00138-9

M3 - Conference article

VL - 261

SP - 179

EP - 209

JO - Theoretical Computer Science

JF - Theoretical Computer Science

SN - 0304-3975

IS - 1

ER -