Improved MDL Estimators Using Local Exponential Family Bundles Applied to Mixture Families

Kohei Miyamoto, Andrew R. Barron, Jun'ichi Takeuchi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The MDL estimators for density estimation, which are defined by two-part codes for universal coding, are analyzed. We give a two-part code for mixture families whose regret is close to the minimax regret, where regret of a code with respect to a target family is the difference between the codelength of the code and the ideal codelength achieved by an element in . Our code is constructed using a probability density in an enlarged family of (a bundle of local exponential families of ) for data description. This result gives a tight upper bound on the risk of the MDL estimator defined by the two-part code, based on the theory introduced by Barron and Cover in 1991.

Original languageEnglish
Title of host publication2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1442-1446
Number of pages5
ISBN (Electronic)9781538692912
DOIs
Publication statusPublished - Jul 2019
Event2019 IEEE International Symposium on Information Theory, ISIT 2019 - Paris, France
Duration: Jul 7 2019Jul 12 2019

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
Volume2019-July
ISSN (Print)2157-8095

Conference

Conference2019 IEEE International Symposium on Information Theory, ISIT 2019
CountryFrance
CityParis
Period7/7/197/12/19

Fingerprint

Data description
Exponential Family
Bundle
Estimator
Regret
Density Estimation
Probability Density
Minimax
Family
Coding
Cover
Upper bound
Target

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Information Systems
  • Modelling and Simulation
  • Applied Mathematics

Cite this

Miyamoto, K., Barron, A. R., & Takeuchi, J. (2019). Improved MDL Estimators Using Local Exponential Family Bundles Applied to Mixture Families. In 2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings (pp. 1442-1446). [8849350] (IEEE International Symposium on Information Theory - Proceedings; Vol. 2019-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT.2019.8849350

Improved MDL Estimators Using Local Exponential Family Bundles Applied to Mixture Families. / Miyamoto, Kohei; Barron, Andrew R.; Takeuchi, Jun'ichi.

2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1442-1446 8849350 (IEEE International Symposium on Information Theory - Proceedings; Vol. 2019-July).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Miyamoto, K, Barron, AR & Takeuchi, J 2019, Improved MDL Estimators Using Local Exponential Family Bundles Applied to Mixture Families. in 2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings., 8849350, IEEE International Symposium on Information Theory - Proceedings, vol. 2019-July, Institute of Electrical and Electronics Engineers Inc., pp. 1442-1446, 2019 IEEE International Symposium on Information Theory, ISIT 2019, Paris, France, 7/7/19. https://doi.org/10.1109/ISIT.2019.8849350
Miyamoto K, Barron AR, Takeuchi J. Improved MDL Estimators Using Local Exponential Family Bundles Applied to Mixture Families. In 2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1442-1446. 8849350. (IEEE International Symposium on Information Theory - Proceedings). https://doi.org/10.1109/ISIT.2019.8849350
Miyamoto, Kohei ; Barron, Andrew R. ; Takeuchi, Jun'ichi. / Improved MDL Estimators Using Local Exponential Family Bundles Applied to Mixture Families. 2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1442-1446 (IEEE International Symposium on Information Theory - Proceedings).
@inproceedings{8a2d8a7519c54038bcf20af58a5a6862,
title = "Improved MDL Estimators Using Local Exponential Family Bundles Applied to Mixture Families",
abstract = "The MDL estimators for density estimation, which are defined by two-part codes for universal coding, are analyzed. We give a two-part code for mixture families whose regret is close to the minimax regret, where regret of a code with respect to a target family is the difference between the codelength of the code and the ideal codelength achieved by an element in . Our code is constructed using a probability density in an enlarged family of (a bundle of local exponential families of ) for data description. This result gives a tight upper bound on the risk of the MDL estimator defined by the two-part code, based on the theory introduced by Barron and Cover in 1991.",
author = "Kohei Miyamoto and Barron, {Andrew R.} and Jun'ichi Takeuchi",
year = "2019",
month = "7",
doi = "10.1109/ISIT.2019.8849350",
language = "English",
series = "IEEE International Symposium on Information Theory - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1442--1446",
booktitle = "2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings",
address = "United States",

}

TY - GEN

T1 - Improved MDL Estimators Using Local Exponential Family Bundles Applied to Mixture Families

AU - Miyamoto, Kohei

AU - Barron, Andrew R.

AU - Takeuchi, Jun'ichi

PY - 2019/7

Y1 - 2019/7

N2 - The MDL estimators for density estimation, which are defined by two-part codes for universal coding, are analyzed. We give a two-part code for mixture families whose regret is close to the minimax regret, where regret of a code with respect to a target family is the difference between the codelength of the code and the ideal codelength achieved by an element in . Our code is constructed using a probability density in an enlarged family of (a bundle of local exponential families of ) for data description. This result gives a tight upper bound on the risk of the MDL estimator defined by the two-part code, based on the theory introduced by Barron and Cover in 1991.

AB - The MDL estimators for density estimation, which are defined by two-part codes for universal coding, are analyzed. We give a two-part code for mixture families whose regret is close to the minimax regret, where regret of a code with respect to a target family is the difference between the codelength of the code and the ideal codelength achieved by an element in . Our code is constructed using a probability density in an enlarged family of (a bundle of local exponential families of ) for data description. This result gives a tight upper bound on the risk of the MDL estimator defined by the two-part code, based on the theory introduced by Barron and Cover in 1991.

UR - http://www.scopus.com/inward/record.url?scp=85073159101&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073159101&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2019.8849350

DO - 10.1109/ISIT.2019.8849350

M3 - Conference contribution

AN - SCOPUS:85073159101

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 1442

EP - 1446

BT - 2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -