TY - GEN

T1 - Some improved sample complexity bounds in the probabilistic PAC learning model

AU - Takeuchi, Jun Ichi

N1 - Publisher Copyright:
© 1993, Springer Verlag. All rights reserved.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.

PY - 1993

Y1 - 1993

N2 - Various authors have proposed probabilistic extensions of Valiant's PAC learning model in which the target to be learned is a conditional (or unconditional) probability distribution. In this paper, we improve upon the best known upper bounds on the sample complexity of learning an important class of stochastic rules called ‘stochastic rules with finite partitioning’ with respect to the classic notion of distance between distributions, the Kullback-Leibler divergence (KL-divergence). In particular, we improve the upper bound of order O(1/e2) due to Abe, Takeuchi, and Warmuth [2] to a bound of order O(1/e). Our proof technique is interesting for at least two reasons: First, previously known upper bounds with respect to the KL-divergence were obtained using the uniform convergence technique, while our improved upper bound is obtained by taking advantage of the properties of the maximum likelihood estimator. Second, our proof relies on the fact that only a linear number of examples are required in order to distinguish a true parametric model from a bad parametric model. The latter notion is apparently related to the notion of discrimination proposed and studied by Yamanishi, but the exact relationship is yet to be determined.

AB - Various authors have proposed probabilistic extensions of Valiant's PAC learning model in which the target to be learned is a conditional (or unconditional) probability distribution. In this paper, we improve upon the best known upper bounds on the sample complexity of learning an important class of stochastic rules called ‘stochastic rules with finite partitioning’ with respect to the classic notion of distance between distributions, the Kullback-Leibler divergence (KL-divergence). In particular, we improve the upper bound of order O(1/e2) due to Abe, Takeuchi, and Warmuth [2] to a bound of order O(1/e). Our proof technique is interesting for at least two reasons: First, previously known upper bounds with respect to the KL-divergence were obtained using the uniform convergence technique, while our improved upper bound is obtained by taking advantage of the properties of the maximum likelihood estimator. Second, our proof relies on the fact that only a linear number of examples are required in order to distinguish a true parametric model from a bad parametric model. The latter notion is apparently related to the notion of discrimination proposed and studied by Yamanishi, but the exact relationship is yet to be determined.

UR - http://www.scopus.com/inward/record.url?scp=79959590702&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79959590702&partnerID=8YFLogxK

U2 - 10.1007/3-540-57369-0_40

DO - 10.1007/3-540-57369-0_40

M3 - Conference contribution

AN - SCOPUS:79959590702

SN - 9783540573692

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 208

EP - 219

BT - Algorithmic Learning Theory - 3rd Workshop, ALT 1992, Proceedings

A2 - Doshita, Shuji

A2 - Furukawa, Koichi

A2 - Jantke, Klaus P.

A2 - Nishida, Toyaki

PB - Springer Verlag

T2 - 3rd Workshop on Algorithmic Learning Theory, ALT 1992

Y2 - 20 October 1992 through 22 October 1992

ER -