Network distributed POMDP with communication

Yuki Iwanari, Yuichi Yabu, Makoto Tasaki, Makoto Yokoo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

While Distributed POMDPs have become popular for modeling multiagent systems in uncertain domains, it is the Network Distributed POMDPs (ND-POMDPs) model that has begun to scale-up the number of agents. The ND-POMDPs can utilize the locality in agents' interactions. However, prior work in ND-POMDPs has failed to address communication. Without communication, the size of a local policy at each agent within the ND-POMDPs grows exponentially in the time horizon. To overcome this problem, we extend existing algorithms so that agents periodically communicate their observation and action histories with each other. After communication, agents can start from new synchronized belief state. Thus, we can avoid the exponential growth in the size of local policies at agents. Furthermore, we introduce an idea that is similar the Point-based Value Iteration algorithm to approximate the value function with a fixed number of representative points. Our experimental results show that we can obtain much longer policies than isting algorithms as long as the interval between communications is small.

Original languageEnglish
Title of host publicationNew Frontiers in Artificial Intelligence - JSAI 2008 Conference and Workshops, Revised Selected Papers
Pages26-38
Number of pages13
DOIs
Publication statusPublished - Jul 23 2009
EventJSAI 2008 Conference and Workshops: New Frontiers in Artificial Intelligence - Asahikawa, Japan
Duration: Jun 11 2008Jun 13 2008

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5447 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherJSAI 2008 Conference and Workshops: New Frontiers in Artificial Intelligence
CountryJapan
CityAsahikawa
Period6/11/086/13/08

Fingerprint

Partially Observable Markov Decision Process
Distributed Networks
Communication
Value Iteration
Scale-up
Exponential Growth
Multi agent systems
Locality
Value Function
Multi-agent Systems
Horizon
Interval
Experimental Results
Interaction
Modeling
Policy

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Iwanari, Y., Yabu, Y., Tasaki, M., & Yokoo, M. (2009). Network distributed POMDP with communication. In New Frontiers in Artificial Intelligence - JSAI 2008 Conference and Workshops, Revised Selected Papers (pp. 26-38). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5447 LNAI). https://doi.org/10.1007/978-3-642-00609-8_4

Network distributed POMDP with communication. / Iwanari, Yuki; Yabu, Yuichi; Tasaki, Makoto; Yokoo, Makoto.

New Frontiers in Artificial Intelligence - JSAI 2008 Conference and Workshops, Revised Selected Papers. 2009. p. 26-38 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5447 LNAI).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Iwanari, Y, Yabu, Y, Tasaki, M & Yokoo, M 2009, Network distributed POMDP with communication. in New Frontiers in Artificial Intelligence - JSAI 2008 Conference and Workshops, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5447 LNAI, pp. 26-38, JSAI 2008 Conference and Workshops: New Frontiers in Artificial Intelligence, Asahikawa, Japan, 6/11/08. https://doi.org/10.1007/978-3-642-00609-8_4
Iwanari Y, Yabu Y, Tasaki M, Yokoo M. Network distributed POMDP with communication. In New Frontiers in Artificial Intelligence - JSAI 2008 Conference and Workshops, Revised Selected Papers. 2009. p. 26-38. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-00609-8_4
Iwanari, Yuki ; Yabu, Yuichi ; Tasaki, Makoto ; Yokoo, Makoto. / Network distributed POMDP with communication. New Frontiers in Artificial Intelligence - JSAI 2008 Conference and Workshops, Revised Selected Papers. 2009. pp. 26-38 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{b14025d601004d01a30f430c2dd885b1,
title = "Network distributed POMDP with communication",
abstract = "While Distributed POMDPs have become popular for modeling multiagent systems in uncertain domains, it is the Network Distributed POMDPs (ND-POMDPs) model that has begun to scale-up the number of agents. The ND-POMDPs can utilize the locality in agents' interactions. However, prior work in ND-POMDPs has failed to address communication. Without communication, the size of a local policy at each agent within the ND-POMDPs grows exponentially in the time horizon. To overcome this problem, we extend existing algorithms so that agents periodically communicate their observation and action histories with each other. After communication, agents can start from new synchronized belief state. Thus, we can avoid the exponential growth in the size of local policies at agents. Furthermore, we introduce an idea that is similar the Point-based Value Iteration algorithm to approximate the value function with a fixed number of representative points. Our experimental results show that we can obtain much longer policies than isting algorithms as long as the interval between communications is small.",
author = "Yuki Iwanari and Yuichi Yabu and Makoto Tasaki and Makoto Yokoo",
year = "2009",
month = "7",
day = "23",
doi = "10.1007/978-3-642-00609-8_4",
language = "English",
isbn = "3642006086",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "26--38",
booktitle = "New Frontiers in Artificial Intelligence - JSAI 2008 Conference and Workshops, Revised Selected Papers",

}

TY - GEN

T1 - Network distributed POMDP with communication

AU - Iwanari, Yuki

AU - Yabu, Yuichi

AU - Tasaki, Makoto

AU - Yokoo, Makoto

PY - 2009/7/23

Y1 - 2009/7/23

N2 - While Distributed POMDPs have become popular for modeling multiagent systems in uncertain domains, it is the Network Distributed POMDPs (ND-POMDPs) model that has begun to scale-up the number of agents. The ND-POMDPs can utilize the locality in agents' interactions. However, prior work in ND-POMDPs has failed to address communication. Without communication, the size of a local policy at each agent within the ND-POMDPs grows exponentially in the time horizon. To overcome this problem, we extend existing algorithms so that agents periodically communicate their observation and action histories with each other. After communication, agents can start from new synchronized belief state. Thus, we can avoid the exponential growth in the size of local policies at agents. Furthermore, we introduce an idea that is similar the Point-based Value Iteration algorithm to approximate the value function with a fixed number of representative points. Our experimental results show that we can obtain much longer policies than isting algorithms as long as the interval between communications is small.

AB - While Distributed POMDPs have become popular for modeling multiagent systems in uncertain domains, it is the Network Distributed POMDPs (ND-POMDPs) model that has begun to scale-up the number of agents. The ND-POMDPs can utilize the locality in agents' interactions. However, prior work in ND-POMDPs has failed to address communication. Without communication, the size of a local policy at each agent within the ND-POMDPs grows exponentially in the time horizon. To overcome this problem, we extend existing algorithms so that agents periodically communicate their observation and action histories with each other. After communication, agents can start from new synchronized belief state. Thus, we can avoid the exponential growth in the size of local policies at agents. Furthermore, we introduce an idea that is similar the Point-based Value Iteration algorithm to approximate the value function with a fixed number of representative points. Our experimental results show that we can obtain much longer policies than isting algorithms as long as the interval between communications is small.

UR - http://www.scopus.com/inward/record.url?scp=67650700513&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67650700513&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-00609-8_4

DO - 10.1007/978-3-642-00609-8_4

M3 - Conference contribution

AN - SCOPUS:67650700513

SN - 3642006086

SN - 9783642006081

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 26

EP - 38

BT - New Frontiers in Artificial Intelligence - JSAI 2008 Conference and Workshops, Revised Selected Papers

ER -