Abstract
Distributed Partially Observable Markov Decision Problems (Distributed POMDPs) are evolving as a popular approach for modeling multiagent systems, and many different algorithms have been proposed to obtain locally or globally optimal policies. Unfortunately, most of these algorithms have either been explicitly designed or experimentally evaluated assuming knowledge of a starting belief point, an assumption that often does not hold in complex, uncertain domains. Instead, in such domains, it is important for agents to explicitly plan over continuous belief spaces. This paper provides a novel algorithm to explicitly compute finite horizon policies over continuous belief spaces, without restricting the space of policies. By marrying an efficient single-agent POMDP solver with a heuristic distributed POMDP policy-generation algorithm, locally optimal joint policies are obtained, each of which dominates within a different part of the belief region. We provide heuristics that significantly improve the efficiency of the resulting algorithm and provide detailed experimental results. To the best of our knowledge, these are the first run-time results for analytically generating policies over continuous belief spaces in distributed POMDPs.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems |
| Pages | 289-296 |
| Number of pages | 8 |
| DOIs | |
| Publication status | Published - Dec 1 2006 |
| Event | Fifth International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS - Hakodate, Japan Duration: May 8 2006 → May 12 2006 |
Publication series
| Name | Proceedings of the International Conference on Autonomous Agents |
|---|---|
| Volume | 2006 |
Other
| Other | Fifth International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS |
|---|---|
| Country | Japan |
| City | Hakodate |
| Period | 5/8/06 → 5/12/06 |
Fingerprint
All Science Journal Classification (ASJC) codes
- Engineering(all)
Cite this
Winning back the CUP for distributed POMDPs : Planning over continuous belief spaces. / Varakantham, Pradeep; Nair, Ranjit; Tambe, Milind; Yokoo, Makoto.
Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems. 2006. p. 289-296 (Proceedings of the International Conference on Autonomous Agents; Vol. 2006).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Winning back the CUP for distributed POMDPs
T2 - Planning over continuous belief spaces
AU - Varakantham, Pradeep
AU - Nair, Ranjit
AU - Tambe, Milind
AU - Yokoo, Makoto
PY - 2006/12/1
Y1 - 2006/12/1
N2 - Distributed Partially Observable Markov Decision Problems (Distributed POMDPs) are evolving as a popular approach for modeling multiagent systems, and many different algorithms have been proposed to obtain locally or globally optimal policies. Unfortunately, most of these algorithms have either been explicitly designed or experimentally evaluated assuming knowledge of a starting belief point, an assumption that often does not hold in complex, uncertain domains. Instead, in such domains, it is important for agents to explicitly plan over continuous belief spaces. This paper provides a novel algorithm to explicitly compute finite horizon policies over continuous belief spaces, without restricting the space of policies. By marrying an efficient single-agent POMDP solver with a heuristic distributed POMDP policy-generation algorithm, locally optimal joint policies are obtained, each of which dominates within a different part of the belief region. We provide heuristics that significantly improve the efficiency of the resulting algorithm and provide detailed experimental results. To the best of our knowledge, these are the first run-time results for analytically generating policies over continuous belief spaces in distributed POMDPs.
AB - Distributed Partially Observable Markov Decision Problems (Distributed POMDPs) are evolving as a popular approach for modeling multiagent systems, and many different algorithms have been proposed to obtain locally or globally optimal policies. Unfortunately, most of these algorithms have either been explicitly designed or experimentally evaluated assuming knowledge of a starting belief point, an assumption that often does not hold in complex, uncertain domains. Instead, in such domains, it is important for agents to explicitly plan over continuous belief spaces. This paper provides a novel algorithm to explicitly compute finite horizon policies over continuous belief spaces, without restricting the space of policies. By marrying an efficient single-agent POMDP solver with a heuristic distributed POMDP policy-generation algorithm, locally optimal joint policies are obtained, each of which dominates within a different part of the belief region. We provide heuristics that significantly improve the efficiency of the resulting algorithm and provide detailed experimental results. To the best of our knowledge, these are the first run-time results for analytically generating policies over continuous belief spaces in distributed POMDPs.
UR - http://www.scopus.com/inward/record.url?scp=34247187490&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=34247187490&partnerID=8YFLogxK
U2 - 10.1145/1160633.1160683
DO - 10.1145/1160633.1160683
M3 - Conference contribution
AN - SCOPUS:34247187490
SN - 1595933034
SN - 9781595933034
T3 - Proceedings of the International Conference on Autonomous Agents
SP - 289
EP - 296
BT - Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems
ER -