Improved sample complexity bounds for parameter estimation

Research output: Contribution to journalConference article

Abstract

Various authors have proposed probabilistic extensions of Valiant's PAC (Probably Approximately Correct) learning model in which the target to be learned is a (conditional) probability distribution. In this paper, we improve upon the best known upper bounds on the sample complexity of the parameter estimation part of the learning problem for distributions and stochastic rules over a finite domain with respect to the Kullback-Leibler divergence (KL-divergence). In particular, we improve the upper bound of order O(1/ε2) due to Abe, Takeuchi, and Warmuth to a bound of order O(1/ε). In obtaining our results, we made use of the properties of a specific estimator (slightly modified maximum likelihood estimator) with respect to the KL-divergence, while previously known upper bounds were obtained using the uniform convergence technique.

Original languageEnglish
Pages (from-to)526-531
Number of pages6
JournalIEICE Transactions on Information and Systems
VolumeE78-D
Issue number5
Publication statusPublished - May 1 1995
Externally publishedYes
EventProceedings of the IEICE Transaction on Information and Systems - Tokyo, Jpn
Duration: Nov 1 1993Nov 1 1993

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Artificial Intelligence

Cite this