Reinforcement learning approach to multi-stage decision making problems with changes in action sets

Takuya Etoh, Hirotaka Takano, Junichi Murata

Research output: Contribution to journalArticle

Abstract

Multi-stage decision making (MSDM) problems often include changes in practical situations. For example, in the shortest route selection problems in road networks, travelling times of road sections vary depending on traffic conditions. The changes give rise to risks in adopting particular solutions to MSDM problems. Therefore, a method is proposed in this paper for solving MSDM problems considering the risks. Reinforcement learning (RL) is adopted as a method for solving those problems, and stochastic changes of action sets are treated. It is necessary to evaluate risks based on subjective views of decision makers (DMs) because the risk evaluation is by nature subjective and depends on DMs. Therefore, an RL approach is proposed which uses a new method for evaluating risks of the changes that can easily incorporate the DM's subjective view and can be readily imbedded in reinforcement learning algorithms. The effectiveness of the method is illustrated with a road network path selection problem.

Original languageEnglish
Pages (from-to)293-299
Number of pages7
JournalArtificial Life and Robotics
Volume17
Issue number2
DOIs
Publication statusPublished - Dec 1 2012

Fingerprint

Reinforcement learning
Decision Making
Decision making
Learning
Learning algorithms
Reinforcement (Psychology)

All Science Journal Classification (ASJC) codes

  • Biochemistry, Genetics and Molecular Biology(all)
  • Artificial Intelligence

Cite this

Reinforcement learning approach to multi-stage decision making problems with changes in action sets. / Etoh, Takuya; Takano, Hirotaka; Murata, Junichi.

In: Artificial Life and Robotics, Vol. 17, No. 2, 01.12.2012, p. 293-299.

Research output: Contribution to journalArticle

@article{0fef182c5c11419fb837fc48c083653d,
title = "Reinforcement learning approach to multi-stage decision making problems with changes in action sets",
abstract = "Multi-stage decision making (MSDM) problems often include changes in practical situations. For example, in the shortest route selection problems in road networks, travelling times of road sections vary depending on traffic conditions. The changes give rise to risks in adopting particular solutions to MSDM problems. Therefore, a method is proposed in this paper for solving MSDM problems considering the risks. Reinforcement learning (RL) is adopted as a method for solving those problems, and stochastic changes of action sets are treated. It is necessary to evaluate risks based on subjective views of decision makers (DMs) because the risk evaluation is by nature subjective and depends on DMs. Therefore, an RL approach is proposed which uses a new method for evaluating risks of the changes that can easily incorporate the DM's subjective view and can be readily imbedded in reinforcement learning algorithms. The effectiveness of the method is illustrated with a road network path selection problem.",
author = "Takuya Etoh and Hirotaka Takano and Junichi Murata",
year = "2012",
month = "12",
day = "1",
doi = "10.1007/s10015-012-0058-9",
language = "English",
volume = "17",
pages = "293--299",
journal = "Artificial Life and Robotics",
issn = "1433-5298",
publisher = "Springer Japan",
number = "2",

}

TY - JOUR

T1 - Reinforcement learning approach to multi-stage decision making problems with changes in action sets

AU - Etoh, Takuya

AU - Takano, Hirotaka

AU - Murata, Junichi

PY - 2012/12/1

Y1 - 2012/12/1

N2 - Multi-stage decision making (MSDM) problems often include changes in practical situations. For example, in the shortest route selection problems in road networks, travelling times of road sections vary depending on traffic conditions. The changes give rise to risks in adopting particular solutions to MSDM problems. Therefore, a method is proposed in this paper for solving MSDM problems considering the risks. Reinforcement learning (RL) is adopted as a method for solving those problems, and stochastic changes of action sets are treated. It is necessary to evaluate risks based on subjective views of decision makers (DMs) because the risk evaluation is by nature subjective and depends on DMs. Therefore, an RL approach is proposed which uses a new method for evaluating risks of the changes that can easily incorporate the DM's subjective view and can be readily imbedded in reinforcement learning algorithms. The effectiveness of the method is illustrated with a road network path selection problem.

AB - Multi-stage decision making (MSDM) problems often include changes in practical situations. For example, in the shortest route selection problems in road networks, travelling times of road sections vary depending on traffic conditions. The changes give rise to risks in adopting particular solutions to MSDM problems. Therefore, a method is proposed in this paper for solving MSDM problems considering the risks. Reinforcement learning (RL) is adopted as a method for solving those problems, and stochastic changes of action sets are treated. It is necessary to evaluate risks based on subjective views of decision makers (DMs) because the risk evaluation is by nature subjective and depends on DMs. Therefore, an RL approach is proposed which uses a new method for evaluating risks of the changes that can easily incorporate the DM's subjective view and can be readily imbedded in reinforcement learning algorithms. The effectiveness of the method is illustrated with a road network path selection problem.

UR - http://www.scopus.com/inward/record.url?scp=84871081762&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84871081762&partnerID=8YFLogxK

U2 - 10.1007/s10015-012-0058-9

DO - 10.1007/s10015-012-0058-9

M3 - Article

AN - SCOPUS:84871081762

VL - 17

SP - 293

EP - 299

JO - Artificial Life and Robotics

JF - Artificial Life and Robotics

SN - 1433-5298

IS - 2

ER -