Information geometry of the family of Markov kernels defined by a context tree

Junnichi Takeuchi, Hiroshi Nagaoka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We prove that a tree model is an exponential family (e-family) of Markov kernels, if and only if it is an FSMX model. The notion of e-family of Markov kernels was first introduced by Nakagawa and Kanaya ('93) in the one-dimensional case. Then, Nagaoka ('05) gave its established form, and Hayashi & Watanabe ('16) discussed it. A tree model is the Markov model defined by a context tree. It is noted by Weinberger et al., ('95) that tree models are classified into two classes; FSMX models and non-FSMX models, depending on the shape of their context trees. The FSMX model is a tree model and a finite state machine. We further show that, for Markov models, the e-family of Markov kernels is equivalent to the asymptotic e-family, which was introduced by Takeuchi & Barron ('98). Note that Takeuchi & Kawabata ('07) proved that non-FSMX tree models are not asymptotic e-families for the binary alphabet case. This paper enhances their result and reveals the information geometrical properties of tree models.

Original languageEnglish
Title of host publication2017 IEEE Information Theory Workshop, ITW 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages429-433
Number of pages5
Volume2018-January
ISBN (Electronic)9781509030972
DOIs
Publication statusPublished - Jan 31 2018
Event2017 IEEE Information Theory Workshop, ITW 2017 - Kaohsiung, Taiwan, Province of China
Duration: Nov 6 2017Nov 10 2017

Other

Other2017 IEEE Information Theory Workshop, ITW 2017
CountryTaiwan, Province of China
CityKaohsiung
Period11/6/1711/10/17

Fingerprint

Information Geometry
kernel
Geometry
Exponential Family
Model
Markov Model
Context
Family
State Machine
Finite automata
Binary
If and only if

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Information Systems
  • Modelling and Simulation
  • Applied Mathematics

Cite this

Takeuchi, J., & Nagaoka, H. (2018). Information geometry of the family of Markov kernels defined by a context tree. In 2017 IEEE Information Theory Workshop, ITW 2017 (Vol. 2018-January, pp. 429-433). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ITW.2017.8278008

Information geometry of the family of Markov kernels defined by a context tree. / Takeuchi, Junnichi; Nagaoka, Hiroshi.

2017 IEEE Information Theory Workshop, ITW 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 429-433.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Takeuchi, J & Nagaoka, H 2018, Information geometry of the family of Markov kernels defined by a context tree. in 2017 IEEE Information Theory Workshop, ITW 2017. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 429-433, 2017 IEEE Information Theory Workshop, ITW 2017, Kaohsiung, Taiwan, Province of China, 11/6/17. https://doi.org/10.1109/ITW.2017.8278008
Takeuchi J, Nagaoka H. Information geometry of the family of Markov kernels defined by a context tree. In 2017 IEEE Information Theory Workshop, ITW 2017. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 429-433 https://doi.org/10.1109/ITW.2017.8278008
Takeuchi, Junnichi ; Nagaoka, Hiroshi. / Information geometry of the family of Markov kernels defined by a context tree. 2017 IEEE Information Theory Workshop, ITW 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 429-433
@inproceedings{215b71675fad447194246ef34f64bb30,
title = "Information geometry of the family of Markov kernels defined by a context tree",
abstract = "We prove that a tree model is an exponential family (e-family) of Markov kernels, if and only if it is an FSMX model. The notion of e-family of Markov kernels was first introduced by Nakagawa and Kanaya ('93) in the one-dimensional case. Then, Nagaoka ('05) gave its established form, and Hayashi & Watanabe ('16) discussed it. A tree model is the Markov model defined by a context tree. It is noted by Weinberger et al., ('95) that tree models are classified into two classes; FSMX models and non-FSMX models, depending on the shape of their context trees. The FSMX model is a tree model and a finite state machine. We further show that, for Markov models, the e-family of Markov kernels is equivalent to the asymptotic e-family, which was introduced by Takeuchi & Barron ('98). Note that Takeuchi & Kawabata ('07) proved that non-FSMX tree models are not asymptotic e-families for the binary alphabet case. This paper enhances their result and reveals the information geometrical properties of tree models.",
author = "Junnichi Takeuchi and Hiroshi Nagaoka",
year = "2018",
month = "1",
day = "31",
doi = "10.1109/ITW.2017.8278008",
language = "English",
volume = "2018-January",
pages = "429--433",
booktitle = "2017 IEEE Information Theory Workshop, ITW 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Information geometry of the family of Markov kernels defined by a context tree

AU - Takeuchi, Junnichi

AU - Nagaoka, Hiroshi

PY - 2018/1/31

Y1 - 2018/1/31

N2 - We prove that a tree model is an exponential family (e-family) of Markov kernels, if and only if it is an FSMX model. The notion of e-family of Markov kernels was first introduced by Nakagawa and Kanaya ('93) in the one-dimensional case. Then, Nagaoka ('05) gave its established form, and Hayashi & Watanabe ('16) discussed it. A tree model is the Markov model defined by a context tree. It is noted by Weinberger et al., ('95) that tree models are classified into two classes; FSMX models and non-FSMX models, depending on the shape of their context trees. The FSMX model is a tree model and a finite state machine. We further show that, for Markov models, the e-family of Markov kernels is equivalent to the asymptotic e-family, which was introduced by Takeuchi & Barron ('98). Note that Takeuchi & Kawabata ('07) proved that non-FSMX tree models are not asymptotic e-families for the binary alphabet case. This paper enhances their result and reveals the information geometrical properties of tree models.

AB - We prove that a tree model is an exponential family (e-family) of Markov kernels, if and only if it is an FSMX model. The notion of e-family of Markov kernels was first introduced by Nakagawa and Kanaya ('93) in the one-dimensional case. Then, Nagaoka ('05) gave its established form, and Hayashi & Watanabe ('16) discussed it. A tree model is the Markov model defined by a context tree. It is noted by Weinberger et al., ('95) that tree models are classified into two classes; FSMX models and non-FSMX models, depending on the shape of their context trees. The FSMX model is a tree model and a finite state machine. We further show that, for Markov models, the e-family of Markov kernels is equivalent to the asymptotic e-family, which was introduced by Takeuchi & Barron ('98). Note that Takeuchi & Kawabata ('07) proved that non-FSMX tree models are not asymptotic e-families for the binary alphabet case. This paper enhances their result and reveals the information geometrical properties of tree models.

UR - http://www.scopus.com/inward/record.url?scp=85046370905&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046370905&partnerID=8YFLogxK

U2 - 10.1109/ITW.2017.8278008

DO - 10.1109/ITW.2017.8278008

M3 - Conference contribution

AN - SCOPUS:85046370905

VL - 2018-January

SP - 429

EP - 433

BT - 2017 IEEE Information Theory Workshop, ITW 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -