Abstract
Various authors have proposed probabilistic extensions of Valiant's PAC (Probably Approximately Correct) learning model in which the target to be learned is a (conditional) probability distribution. In this paper, we improve upon the best known upper bounds on the sample complexity of the parameter estimation part of the learning problem for distributions and stochastic rules over a finite domain with respect to the Kullback-Leibler divergence (KL-divergence). In particular, we improve the upper bound of order O(1/ε2) due to Abe, Takeuchi, and Warmuth to a bound of order O(1/ε). In obtaining our results, we made use of the properties of a specific estimator (slightly modified maximum likelihood estimator) with respect to the KL-divergence, while previously known upper bounds were obtained using the uniform convergence technique.
Original language | English |
---|---|
Pages (from-to) | 526-531 |
Number of pages | 6 |
Journal | IEICE Transactions on Information and Systems |
Volume | E78-D |
Issue number | 5 |
Publication status | Published - May 1 1995 |
Externally published | Yes |
Event | Proceedings of the IEICE Transaction on Information and Systems - Tokyo, Jpn Duration: Nov 1 1993 → Nov 1 1993 |
All Science Journal Classification (ASJC) codes
- Software
- Hardware and Architecture
- Computer Vision and Pattern Recognition
- Electrical and Electronic Engineering
- Artificial Intelligence