Asymptotic Behavior of Typical Sets and the Smallest High Probability Set

Munenori Eto, Masanori Kawakita, Junnichi Takeuchi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Cardinality of typical sets and the smallest high probability set for discrete memoryless sources and stationary Markov sources are considered. Usually, its width is fixed in the definition of both sets, but sometimes it is assumed that the width converges to zero as the length of sequence n goes to infinity. In such setting, some condition for the width is necessary to make the cardinality near 2nH, where H denotes the entropy rate of the information source. In this article, we give sufficient conditions for the above propositions. We also show sufficient conditions for that they don't hold for DMS and Markov sources under a certain restriction.

Original languageEnglish
Title of host publicationProceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages717-721
Number of pages5
ISBN (Electronic)9784885523182
DOIs
Publication statusPublished - Mar 8 2019
Event15th International Symposium on Information Theory and Its Applications, ISITA 2018 - Singapore, Singapore
Duration: Oct 28 2018Oct 31 2018

Publication series

NameProceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018

Conference

Conference15th International Symposium on Information Theory and Its Applications, ISITA 2018
CountrySingapore
CitySingapore
Period10/28/1810/31/18

Fingerprint

Entropy

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Information Systems

Cite this

Eto, M., Kawakita, M., & Takeuchi, J. (2019). Asymptotic Behavior of Typical Sets and the Smallest High Probability Set. In Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018 (pp. 717-721). [8664327] (Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/ISITA.2018.8664327

Asymptotic Behavior of Typical Sets and the Smallest High Probability Set. / Eto, Munenori; Kawakita, Masanori; Takeuchi, Junnichi.

Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018. Institute of Electrical and Electronics Engineers Inc., 2019. p. 717-721 8664327 (Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Eto, M, Kawakita, M & Takeuchi, J 2019, Asymptotic Behavior of Typical Sets and the Smallest High Probability Set. in Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018., 8664327, Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018, Institute of Electrical and Electronics Engineers Inc., pp. 717-721, 15th International Symposium on Information Theory and Its Applications, ISITA 2018, Singapore, Singapore, 10/28/18. https://doi.org/10.23919/ISITA.2018.8664327
Eto M, Kawakita M, Takeuchi J. Asymptotic Behavior of Typical Sets and the Smallest High Probability Set. In Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018. Institute of Electrical and Electronics Engineers Inc. 2019. p. 717-721. 8664327. (Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018). https://doi.org/10.23919/ISITA.2018.8664327
Eto, Munenori ; Kawakita, Masanori ; Takeuchi, Junnichi. / Asymptotic Behavior of Typical Sets and the Smallest High Probability Set. Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 717-721 (Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018).
@inproceedings{476fce7a5a114f698b2b4a97b1abc90d,
title = "Asymptotic Behavior of Typical Sets and the Smallest High Probability Set",
abstract = "Cardinality of typical sets and the smallest high probability set for discrete memoryless sources and stationary Markov sources are considered. Usually, its width is fixed in the definition of both sets, but sometimes it is assumed that the width converges to zero as the length of sequence n goes to infinity. In such setting, some condition for the width is necessary to make the cardinality near 2nH, where H denotes the entropy rate of the information source. In this article, we give sufficient conditions for the above propositions. We also show sufficient conditions for that they don't hold for DMS and Markov sources under a certain restriction.",
author = "Munenori Eto and Masanori Kawakita and Junnichi Takeuchi",
year = "2019",
month = "3",
day = "8",
doi = "10.23919/ISITA.2018.8664327",
language = "English",
series = "Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "717--721",
booktitle = "Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018",
address = "United States",

}

TY - GEN

T1 - Asymptotic Behavior of Typical Sets and the Smallest High Probability Set

AU - Eto, Munenori

AU - Kawakita, Masanori

AU - Takeuchi, Junnichi

PY - 2019/3/8

Y1 - 2019/3/8

N2 - Cardinality of typical sets and the smallest high probability set for discrete memoryless sources and stationary Markov sources are considered. Usually, its width is fixed in the definition of both sets, but sometimes it is assumed that the width converges to zero as the length of sequence n goes to infinity. In such setting, some condition for the width is necessary to make the cardinality near 2nH, where H denotes the entropy rate of the information source. In this article, we give sufficient conditions for the above propositions. We also show sufficient conditions for that they don't hold for DMS and Markov sources under a certain restriction.

AB - Cardinality of typical sets and the smallest high probability set for discrete memoryless sources and stationary Markov sources are considered. Usually, its width is fixed in the definition of both sets, but sometimes it is assumed that the width converges to zero as the length of sequence n goes to infinity. In such setting, some condition for the width is necessary to make the cardinality near 2nH, where H denotes the entropy rate of the information source. In this article, we give sufficient conditions for the above propositions. We also show sufficient conditions for that they don't hold for DMS and Markov sources under a certain restriction.

UR - http://www.scopus.com/inward/record.url?scp=85063891403&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063891403&partnerID=8YFLogxK

U2 - 10.23919/ISITA.2018.8664327

DO - 10.23919/ISITA.2018.8664327

M3 - Conference contribution

T3 - Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018

SP - 717

EP - 721

BT - Proceedings of 2018 International Symposium on Information Theory and Its Applications, ISITA 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -