Communication for improving policy computation in distributed POMDPs

Ranjit Nair, Milind Tambe, Maayan Roth, Makoto Yokoo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

46 Citations (Scopus)

Abstract

Distributed Partially Observable Marfcov Decision Problems (POMDPs) are emerging as a popular approach for modeling multiagent teamwork where a group of agents work together to jointly maximize a reward function. Since the problem of finding the optimal joint policy for a distributed POMDP has been shown to be NEXP-Complete if no assumptions are made about the domain conditions, several locally optimal approaches have emerged as a viable solution. However, the use of communicative actions as part of these locally optimal algorithms has been largely ignored or has been applied only under restrictive assumptions about the domain. In this paper, we show how communicative acts can be explicitly introduced in order to find locally optimal joint policies that allow agents to coordinate better through synchronization achieved via communication. Furthermore, the introduction of communication allows us to develop a novel compact policy representation that results in savings of both space and time which are verified empirically. Finally, through the imposition of constraints on communication such as not going without communicating for more than K steps, even greater space and time savings can be obtained.

Original languageEnglish
Title of host publicationProceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004
EditorsN.R. Jennings, C. Sierra, L. Sonenberg, M. Tambe
Pages1098-1105
Number of pages8
Publication statusPublished - Sep 27 2004
EventProceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004 - New York, NY, United States
Duration: Jul 19 2004Jul 23 2004

Publication series

NameProceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004
Volume3

Other

OtherProceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004
CountryUnited States
CityNew York, NY
Period7/19/047/23/04

Fingerprint

Communication
Synchronization

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Cite this

Nair, R., Tambe, M., Roth, M., & Yokoo, M. (2004). Communication for improving policy computation in distributed POMDPs. In N. R. Jennings, C. Sierra, L. Sonenberg, & M. Tambe (Eds.), Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004 (pp. 1098-1105). (Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004; Vol. 3).

Communication for improving policy computation in distributed POMDPs. / Nair, Ranjit; Tambe, Milind; Roth, Maayan; Yokoo, Makoto.

Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004. ed. / N.R. Jennings; C. Sierra; L. Sonenberg; M. Tambe. 2004. p. 1098-1105 (Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004; Vol. 3).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nair, R, Tambe, M, Roth, M & Yokoo, M 2004, Communication for improving policy computation in distributed POMDPs. in NR Jennings, C Sierra, L Sonenberg & M Tambe (eds), Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004. Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004, vol. 3, pp. 1098-1105, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004, New York, NY, United States, 7/19/04.
Nair R, Tambe M, Roth M, Yokoo M. Communication for improving policy computation in distributed POMDPs. In Jennings NR, Sierra C, Sonenberg L, Tambe M, editors, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004. 2004. p. 1098-1105. (Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004).
Nair, Ranjit ; Tambe, Milind ; Roth, Maayan ; Yokoo, Makoto. / Communication for improving policy computation in distributed POMDPs. Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004. editor / N.R. Jennings ; C. Sierra ; L. Sonenberg ; M. Tambe. 2004. pp. 1098-1105 (Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004).
@inproceedings{1830ac891c554961a525aef1ab19505c,
title = "Communication for improving policy computation in distributed POMDPs",
abstract = "Distributed Partially Observable Marfcov Decision Problems (POMDPs) are emerging as a popular approach for modeling multiagent teamwork where a group of agents work together to jointly maximize a reward function. Since the problem of finding the optimal joint policy for a distributed POMDP has been shown to be NEXP-Complete if no assumptions are made about the domain conditions, several locally optimal approaches have emerged as a viable solution. However, the use of communicative actions as part of these locally optimal algorithms has been largely ignored or has been applied only under restrictive assumptions about the domain. In this paper, we show how communicative acts can be explicitly introduced in order to find locally optimal joint policies that allow agents to coordinate better through synchronization achieved via communication. Furthermore, the introduction of communication allows us to develop a novel compact policy representation that results in savings of both space and time which are verified empirically. Finally, through the imposition of constraints on communication such as not going without communicating for more than K steps, even greater space and time savings can be obtained.",
author = "Ranjit Nair and Milind Tambe and Maayan Roth and Makoto Yokoo",
year = "2004",
month = "9",
day = "27",
language = "English",
isbn = "1581138644",
series = "Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004",
pages = "1098--1105",
editor = "N.R. Jennings and C. Sierra and L. Sonenberg and M. Tambe",
booktitle = "Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004",

}

TY - GEN

T1 - Communication for improving policy computation in distributed POMDPs

AU - Nair, Ranjit

AU - Tambe, Milind

AU - Roth, Maayan

AU - Yokoo, Makoto

PY - 2004/9/27

Y1 - 2004/9/27

N2 - Distributed Partially Observable Marfcov Decision Problems (POMDPs) are emerging as a popular approach for modeling multiagent teamwork where a group of agents work together to jointly maximize a reward function. Since the problem of finding the optimal joint policy for a distributed POMDP has been shown to be NEXP-Complete if no assumptions are made about the domain conditions, several locally optimal approaches have emerged as a viable solution. However, the use of communicative actions as part of these locally optimal algorithms has been largely ignored or has been applied only under restrictive assumptions about the domain. In this paper, we show how communicative acts can be explicitly introduced in order to find locally optimal joint policies that allow agents to coordinate better through synchronization achieved via communication. Furthermore, the introduction of communication allows us to develop a novel compact policy representation that results in savings of both space and time which are verified empirically. Finally, through the imposition of constraints on communication such as not going without communicating for more than K steps, even greater space and time savings can be obtained.

AB - Distributed Partially Observable Marfcov Decision Problems (POMDPs) are emerging as a popular approach for modeling multiagent teamwork where a group of agents work together to jointly maximize a reward function. Since the problem of finding the optimal joint policy for a distributed POMDP has been shown to be NEXP-Complete if no assumptions are made about the domain conditions, several locally optimal approaches have emerged as a viable solution. However, the use of communicative actions as part of these locally optimal algorithms has been largely ignored or has been applied only under restrictive assumptions about the domain. In this paper, we show how communicative acts can be explicitly introduced in order to find locally optimal joint policies that allow agents to coordinate better through synchronization achieved via communication. Furthermore, the introduction of communication allows us to develop a novel compact policy representation that results in savings of both space and time which are verified empirically. Finally, through the imposition of constraints on communication such as not going without communicating for more than K steps, even greater space and time savings can be obtained.

UR - http://www.scopus.com/inward/record.url?scp=4544315369&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=4544315369&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:4544315369

SN - 1581138644

SN - 9781581138641

T3 - Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004

SP - 1098

EP - 1105

BT - Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004

A2 - Jennings, N.R.

A2 - Sierra, C.

A2 - Sonenberg, L.

A2 - Tambe, M.

ER -