Multiagent planning with trembling-hand perfect equilibrium in multiagent POMDPs

Yuichi Yabu, Makoto Yokoo, Atsushi Iwasaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Multiagent Partially Observable Markov Decision Processes are a popular model of multiagent systems with uncertainty. Since the computational cost for finding an optimal joint policy is prohibitive, a Joint Equilibrium-based Search for Policies with Nash Equilibrium (JESP-NE) is proposed that finds a locally optimal joint policy in which each policy is a best response to other policies; i.e., the joint policy is a Nash equilibrium. One limitation of JESP-NE is that the quality of the obtained joint policy depends on the predefined default policy. More specifically, when finding a best response, if some observation have zero probabilities, JESP-NE uses this default policy. If the default policy is quite bad, JESP-NE tends to converge to a sub-optimal joint policy. In this paper, we propose a method that finds a locally optimal joint policy based on a concept called Trembling-hand Perfect Equilibrium (TPE). In finding a TPE, we assume that an agent might make a mistake in selecting its action with small probability. Thus, an observation with zero probability in JESP-NE will have non-zero probability. We no longer use the default policy. As a result, JESP-TPE can converge to a better joint policy than the JESP-NE, which we confirm this fact by experimental evaluations.

Original languageEnglish
Title of host publicationAgent Computing and Multi-Agent Systems - 10th Pacific Rim International Conference on Multi-Agents, PRIMA 2007, Revised Papers
Pages13-24
Number of pages12
DOIs
Publication statusPublished - Jul 17 2009
Event10th Pacific Rim International Conference on Multi-Agents, PRIMA 2007 - Bangkok, Thailand
Duration: Nov 21 2007Nov 23 2007

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5044 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other10th Pacific Rim International Conference on Multi-Agents, PRIMA 2007
CountryThailand
CityBangkok
Period11/21/0711/23/07

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Yabu, Y., Yokoo, M., & Iwasaki, A. (2009). Multiagent planning with trembling-hand perfect equilibrium in multiagent POMDPs. In Agent Computing and Multi-Agent Systems - 10th Pacific Rim International Conference on Multi-Agents, PRIMA 2007, Revised Papers (pp. 13-24). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5044 LNAI). https://doi.org/10.1007/978-3-642-01639-4_2