TD algorithm for the variance of return and mean-variance reinforcement learning

Makoto Sato, Hajime Kimura, Shibenobu Kobayashi

Research output: Contribution to journalArticlepeer-review

31 Citations (Scopus)


Estimating probability distributions on returns provides various sophisticated decision making schemes for control problems in Markov environments, including risk-sensitive control, efficient exploration of environments and so on. Many reinforcement learning algorithms, however, have simply relied on the expected return. This paper provides a scheme of decision making using mean and variance of returndistributions. This paper presents a TD algorithm for estimating the variance of return in MDP(Markov decision processes) environments and a gradient-based reinforcement learning algorithm on the variance penalized criterion, which is a typical criterion in risk-avoiding control. Empirical results demonstrate behaviors of the algorithms and validates of the criterion for risk-avoiding sequential decision tasks.

Original languageEnglish
Pages (from-to)353-362
Number of pages10
JournalTransactions of the Japanese Society for Artificial Intelligence
Issue number3
Publication statusPublished - 2001
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence


Dive into the research topics of 'TD algorithm for the variance of return and mean-variance reinforcement learning'. Together they form a unique fingerprint.

Cite this