Introducing communication in Dis-POMDPs with Finite State Machines

Yuki Iwanari, Makoto Tasaki, Makoto Yokoo, Atsushi Iwasaki, Yuko Sakurai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Distributed Partially Observable Markov Decision Problems (Dis-POMDPs) are emerging as a popular approach for modeling sequential decision making in teams operating under uncertainty. To achieve coherent behaviors of agents, it is essential to perform appropriate run-time communication. Thus, there have been many works on the run-time communication schemes in Dis-POMDPs. Also, a Finite State Machine (FSM) is a popular representation for describing a local policy that works in a very long or infinite time horizon. In this paper, we examine a run-time communication scheme when the local policy of each agent is represented as an FSM. In this scheme, the meaning of each message is not predefined; it is given implicitly by the interaction between local policies. We propose an iterative-improvement type algorithm that searches for a joint policy where run-time communication incurs some cost. Thus, agents use run-time communication only when doing so is cost-effective. Interestingly, our algorithm can find a joint policy that obtains a better expected reward than a hand-crafted joint policy, and it requires fewer nodes in the local FSM and fewer message types. Furthermore, we experimentally show that our algorithm can obtain a joint policy that consists of sufficiently complex local FSMs within a reasonable amount of time.

Original languageEnglish
Title of host publicationProceedings - 2009 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT 2009
Pages267-270
Number of pages4
DOIs
Publication statusPublished - Dec 1 2009
Event2009 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2009 - Milano, Italy
Duration: Sep 15 2009Sep 18 2009

Publication series

NameProceedings - 2009 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT 2009
Volume2

Other

Other2009 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2009
CountryItaly
CityMilano
Period9/15/099/18/09

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Software

Cite this

Iwanari, Y., Tasaki, M., Yokoo, M., Iwasaki, A., & Sakurai, Y. (2009). Introducing communication in Dis-POMDPs with Finite State Machines. In Proceedings - 2009 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT 2009 (pp. 267-270). [5284831] (Proceedings - 2009 IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT 2009; Vol. 2). https://doi.org/10.1109/WI-IAT.2009.161