Winning back the CUP for distributed POMDPs: Planning over continuous belief spaces

Pradeep Varakantham, Ranjit Nair, Milind Tambe, Makoto Yokoo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Distributed Partially Observable Markov Decision Problems (Distributed POMDPs) are evolving as a popular approach for modeling multiagent systems, and many different algorithms have been proposed to obtain locally or globally optimal policies. Unfortunately, most of these algorithms have either been explicitly designed or experimentally evaluated assuming knowledge of a starting belief point, an assumption that often does not hold in complex, uncertain domains. Instead, in such domains, it is important for agents to explicitly plan over continuous belief spaces. This paper provides a novel algorithm to explicitly compute finite horizon policies over continuous belief spaces, without restricting the space of policies. By marrying an efficient single-agent POMDP solver with a heuristic distributed POMDP policy-generation algorithm, locally optimal joint policies are obtained, each of which dominates within a different part of the belief region. We provide heuristics that significantly improve the efficiency of the resulting algorithm and provide detailed experimental results. To the best of our knowledge, these are the first run-time results for analytically generating policies over continuous belief spaces in distributed POMDPs.

Original languageEnglish
Title of host publicationProceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems
Pages289-296
Number of pages8
DOIs
Publication statusPublished - Dec 1 2006
EventFifth International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS - Hakodate, Japan
Duration: May 8 2006May 12 2006

Publication series

NameProceedings of the International Conference on Autonomous Agents
Volume2006

Other

OtherFifth International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
CountryJapan
CityHakodate
Period5/8/065/12/06

Fingerprint

Planning
Multi agent systems

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Cite this

Varakantham, P., Nair, R., Tambe, M., & Yokoo, M. (2006). Winning back the CUP for distributed POMDPs: Planning over continuous belief spaces. In Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems (pp. 289-296). (Proceedings of the International Conference on Autonomous Agents; Vol. 2006). https://doi.org/10.1145/1160633.1160683

Winning back the CUP for distributed POMDPs : Planning over continuous belief spaces. / Varakantham, Pradeep; Nair, Ranjit; Tambe, Milind; Yokoo, Makoto.

Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems. 2006. p. 289-296 (Proceedings of the International Conference on Autonomous Agents; Vol. 2006).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Varakantham, P, Nair, R, Tambe, M & Yokoo, M 2006, Winning back the CUP for distributed POMDPs: Planning over continuous belief spaces. in Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems. Proceedings of the International Conference on Autonomous Agents, vol. 2006, pp. 289-296, Fifth International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, Hakodate, Japan, 5/8/06. https://doi.org/10.1145/1160633.1160683
Varakantham P, Nair R, Tambe M, Yokoo M. Winning back the CUP for distributed POMDPs: Planning over continuous belief spaces. In Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems. 2006. p. 289-296. (Proceedings of the International Conference on Autonomous Agents). https://doi.org/10.1145/1160633.1160683
Varakantham, Pradeep ; Nair, Ranjit ; Tambe, Milind ; Yokoo, Makoto. / Winning back the CUP for distributed POMDPs : Planning over continuous belief spaces. Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems. 2006. pp. 289-296 (Proceedings of the International Conference on Autonomous Agents).
@inproceedings{8154efaa6fcc41b197642986472567d8,
title = "Winning back the CUP for distributed POMDPs: Planning over continuous belief spaces",
abstract = "Distributed Partially Observable Markov Decision Problems (Distributed POMDPs) are evolving as a popular approach for modeling multiagent systems, and many different algorithms have been proposed to obtain locally or globally optimal policies. Unfortunately, most of these algorithms have either been explicitly designed or experimentally evaluated assuming knowledge of a starting belief point, an assumption that often does not hold in complex, uncertain domains. Instead, in such domains, it is important for agents to explicitly plan over continuous belief spaces. This paper provides a novel algorithm to explicitly compute finite horizon policies over continuous belief spaces, without restricting the space of policies. By marrying an efficient single-agent POMDP solver with a heuristic distributed POMDP policy-generation algorithm, locally optimal joint policies are obtained, each of which dominates within a different part of the belief region. We provide heuristics that significantly improve the efficiency of the resulting algorithm and provide detailed experimental results. To the best of our knowledge, these are the first run-time results for analytically generating policies over continuous belief spaces in distributed POMDPs.",
author = "Pradeep Varakantham and Ranjit Nair and Milind Tambe and Makoto Yokoo",
year = "2006",
month = "12",
day = "1",
doi = "10.1145/1160633.1160683",
language = "English",
isbn = "1595933034",
series = "Proceedings of the International Conference on Autonomous Agents",
pages = "289--296",
booktitle = "Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems",

}

TY - GEN

T1 - Winning back the CUP for distributed POMDPs

T2 - Planning over continuous belief spaces

AU - Varakantham, Pradeep

AU - Nair, Ranjit

AU - Tambe, Milind

AU - Yokoo, Makoto

PY - 2006/12/1

Y1 - 2006/12/1

N2 - Distributed Partially Observable Markov Decision Problems (Distributed POMDPs) are evolving as a popular approach for modeling multiagent systems, and many different algorithms have been proposed to obtain locally or globally optimal policies. Unfortunately, most of these algorithms have either been explicitly designed or experimentally evaluated assuming knowledge of a starting belief point, an assumption that often does not hold in complex, uncertain domains. Instead, in such domains, it is important for agents to explicitly plan over continuous belief spaces. This paper provides a novel algorithm to explicitly compute finite horizon policies over continuous belief spaces, without restricting the space of policies. By marrying an efficient single-agent POMDP solver with a heuristic distributed POMDP policy-generation algorithm, locally optimal joint policies are obtained, each of which dominates within a different part of the belief region. We provide heuristics that significantly improve the efficiency of the resulting algorithm and provide detailed experimental results. To the best of our knowledge, these are the first run-time results for analytically generating policies over continuous belief spaces in distributed POMDPs.

AB - Distributed Partially Observable Markov Decision Problems (Distributed POMDPs) are evolving as a popular approach for modeling multiagent systems, and many different algorithms have been proposed to obtain locally or globally optimal policies. Unfortunately, most of these algorithms have either been explicitly designed or experimentally evaluated assuming knowledge of a starting belief point, an assumption that often does not hold in complex, uncertain domains. Instead, in such domains, it is important for agents to explicitly plan over continuous belief spaces. This paper provides a novel algorithm to explicitly compute finite horizon policies over continuous belief spaces, without restricting the space of policies. By marrying an efficient single-agent POMDP solver with a heuristic distributed POMDP policy-generation algorithm, locally optimal joint policies are obtained, each of which dominates within a different part of the belief region. We provide heuristics that significantly improve the efficiency of the resulting algorithm and provide detailed experimental results. To the best of our knowledge, these are the first run-time results for analytically generating policies over continuous belief spaces in distributed POMDPs.

UR - http://www.scopus.com/inward/record.url?scp=34247187490&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34247187490&partnerID=8YFLogxK

U2 - 10.1145/1160633.1160683

DO - 10.1145/1160633.1160683

M3 - Conference contribution

AN - SCOPUS:34247187490

SN - 1595933034

SN - 9781595933034

T3 - Proceedings of the International Conference on Autonomous Agents

SP - 289

EP - 296

BT - Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems

ER -