Taming decentralized POMDPs

Towards efficient policy computation for multiagent settings

R. Nair, M. Tambe, Makoto Yokoo, D. Pynadath, S. Marsella

Research output: Contribution to journalConference article

209 Citations (Scopus)

Abstract

The problem of deriving joint policies for a group of agents that maximize some joint reward function can be modeled as a decentralized partially observable Markov decision process (POMDP). Yet, despite the growing importance and applications of decentralized POMDP models in the multiagents arena, few algorithms have been developed for efficiently deriving joint policies for these models. This paper presents a new class of locally optimal algorithms called "Joint Equilibrium-based search for policies (JESP)". We first describe an exhaustive version of JESP and subsequently a novel dynamic programming approach to JESP. Our complexity analysis reveals the potential for exponential speedups due to the dynamic programming approach. These theoretical results are verified via empirical comparisons of the two JESP versions with each other and with a globally optimal brute-force search algorithm. Finally, we prove piece-wise linear and convexity (PWLC) properties, thus taking steps towards developing algorithms for continuous belief states.

Original languageEnglish
Pages (from-to)705-711
Number of pages7
JournalIJCAI International Joint Conference on Artificial Intelligence
Publication statusPublished - Dec 1 2003
Externally publishedYes
Event18th International Joint Conference on Artificial Intelligence, IJCAI 2003 - Acapulco, Mexico
Duration: Aug 9 2003Aug 15 2003

Fingerprint

Dynamic programming
policy
decision process

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence

Cite this

Taming decentralized POMDPs : Towards efficient policy computation for multiagent settings. / Nair, R.; Tambe, M.; Yokoo, Makoto; Pynadath, D.; Marsella, S.

In: IJCAI International Joint Conference on Artificial Intelligence, 01.12.2003, p. 705-711.

Research output: Contribution to journalConference article

@article{369888bbb7e848a0851994e4f279d354,
title = "Taming decentralized POMDPs: Towards efficient policy computation for multiagent settings",
abstract = "The problem of deriving joint policies for a group of agents that maximize some joint reward function can be modeled as a decentralized partially observable Markov decision process (POMDP). Yet, despite the growing importance and applications of decentralized POMDP models in the multiagents arena, few algorithms have been developed for efficiently deriving joint policies for these models. This paper presents a new class of locally optimal algorithms called {"}Joint Equilibrium-based search for policies (JESP){"}. We first describe an exhaustive version of JESP and subsequently a novel dynamic programming approach to JESP. Our complexity analysis reveals the potential for exponential speedups due to the dynamic programming approach. These theoretical results are verified via empirical comparisons of the two JESP versions with each other and with a globally optimal brute-force search algorithm. Finally, we prove piece-wise linear and convexity (PWLC) properties, thus taking steps towards developing algorithms for continuous belief states.",
author = "R. Nair and M. Tambe and Makoto Yokoo and D. Pynadath and S. Marsella",
year = "2003",
month = "12",
day = "1",
language = "English",
pages = "705--711",
journal = "IJCAI International Joint Conference on Artificial Intelligence",
issn = "1045-0823",

}

TY - JOUR

T1 - Taming decentralized POMDPs

T2 - Towards efficient policy computation for multiagent settings

AU - Nair, R.

AU - Tambe, M.

AU - Yokoo, Makoto

AU - Pynadath, D.

AU - Marsella, S.

PY - 2003/12/1

Y1 - 2003/12/1

N2 - The problem of deriving joint policies for a group of agents that maximize some joint reward function can be modeled as a decentralized partially observable Markov decision process (POMDP). Yet, despite the growing importance and applications of decentralized POMDP models in the multiagents arena, few algorithms have been developed for efficiently deriving joint policies for these models. This paper presents a new class of locally optimal algorithms called "Joint Equilibrium-based search for policies (JESP)". We first describe an exhaustive version of JESP and subsequently a novel dynamic programming approach to JESP. Our complexity analysis reveals the potential for exponential speedups due to the dynamic programming approach. These theoretical results are verified via empirical comparisons of the two JESP versions with each other and with a globally optimal brute-force search algorithm. Finally, we prove piece-wise linear and convexity (PWLC) properties, thus taking steps towards developing algorithms for continuous belief states.

AB - The problem of deriving joint policies for a group of agents that maximize some joint reward function can be modeled as a decentralized partially observable Markov decision process (POMDP). Yet, despite the growing importance and applications of decentralized POMDP models in the multiagents arena, few algorithms have been developed for efficiently deriving joint policies for these models. This paper presents a new class of locally optimal algorithms called "Joint Equilibrium-based search for policies (JESP)". We first describe an exhaustive version of JESP and subsequently a novel dynamic programming approach to JESP. Our complexity analysis reveals the potential for exponential speedups due to the dynamic programming approach. These theoretical results are verified via empirical comparisons of the two JESP versions with each other and with a globally optimal brute-force search algorithm. Finally, we prove piece-wise linear and convexity (PWLC) properties, thus taking steps towards developing algorithms for continuous belief states.

UR - http://www.scopus.com/inward/record.url?scp=84880823326&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84880823326&partnerID=8YFLogxK

M3 - Conference article

SP - 705

EP - 711

JO - IJCAI International Joint Conference on Artificial Intelligence

JF - IJCAI International Joint Conference on Artificial Intelligence

SN - 1045-0823

ER -