Natural gradient actor-critic algorithms using random rectangular coarse coding

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Learning performance of natural gradient actor-critic algorithms is outstanding especially in high-dimensional spaces than conventional actor-critic algorithms. However, representation issues of stochastic policies or value functions are remaining because the actor-critic approaches need to design it carefully. The author has proposed random rectangular coarse coding, that is very simple and suited for approximating Q-values in high-dimensional state-action space. This paper shows a quantitative analysis of the random coarse coding comparing with regular-grid approaches, and presents a new approach that combines the natural gradient actor-critic with the random rectangular coarse coding.

Original languageEnglish
Title of host publicationProceedings of SICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology
Pages2027-2034
Number of pages8
DOIs
Publication statusPublished - 2008
EventSICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology - Tokyo, Japan
Duration: Aug 20 2008Aug 22 2008

Other

OtherSICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology
CountryJapan
CityTokyo
Period8/20/088/22/08

Fingerprint

Chemical analysis

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering
  • Control and Systems Engineering
  • Computer Science Applications

Cite this

Kimura, H. (2008). Natural gradient actor-critic algorithms using random rectangular coarse coding. In Proceedings of SICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology (pp. 2027-2034). [4654995] https://doi.org/10.1109/SICE.2008.4654995

Natural gradient actor-critic algorithms using random rectangular coarse coding. / Kimura, Hajime.

Proceedings of SICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology. 2008. p. 2027-2034 4654995.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kimura, H 2008, Natural gradient actor-critic algorithms using random rectangular coarse coding. in Proceedings of SICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology., 4654995, pp. 2027-2034, SICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology, Tokyo, Japan, 8/20/08. https://doi.org/10.1109/SICE.2008.4654995
Kimura H. Natural gradient actor-critic algorithms using random rectangular coarse coding. In Proceedings of SICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology. 2008. p. 2027-2034. 4654995 https://doi.org/10.1109/SICE.2008.4654995
Kimura, Hajime. / Natural gradient actor-critic algorithms using random rectangular coarse coding. Proceedings of SICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology. 2008. pp. 2027-2034
@inproceedings{b904b04201ff4023b6aa49ecbf3090f4,
title = "Natural gradient actor-critic algorithms using random rectangular coarse coding",
abstract = "Learning performance of natural gradient actor-critic algorithms is outstanding especially in high-dimensional spaces than conventional actor-critic algorithms. However, representation issues of stochastic policies or value functions are remaining because the actor-critic approaches need to design it carefully. The author has proposed random rectangular coarse coding, that is very simple and suited for approximating Q-values in high-dimensional state-action space. This paper shows a quantitative analysis of the random coarse coding comparing with regular-grid approaches, and presents a new approach that combines the natural gradient actor-critic with the random rectangular coarse coding.",
author = "Hajime Kimura",
year = "2008",
doi = "10.1109/SICE.2008.4654995",
language = "English",
isbn = "9784907764296",
pages = "2027--2034",
booktitle = "Proceedings of SICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology",

}

TY - GEN

T1 - Natural gradient actor-critic algorithms using random rectangular coarse coding

AU - Kimura, Hajime

PY - 2008

Y1 - 2008

N2 - Learning performance of natural gradient actor-critic algorithms is outstanding especially in high-dimensional spaces than conventional actor-critic algorithms. However, representation issues of stochastic policies or value functions are remaining because the actor-critic approaches need to design it carefully. The author has proposed random rectangular coarse coding, that is very simple and suited for approximating Q-values in high-dimensional state-action space. This paper shows a quantitative analysis of the random coarse coding comparing with regular-grid approaches, and presents a new approach that combines the natural gradient actor-critic with the random rectangular coarse coding.

AB - Learning performance of natural gradient actor-critic algorithms is outstanding especially in high-dimensional spaces than conventional actor-critic algorithms. However, representation issues of stochastic policies or value functions are remaining because the actor-critic approaches need to design it carefully. The author has proposed random rectangular coarse coding, that is very simple and suited for approximating Q-values in high-dimensional state-action space. This paper shows a quantitative analysis of the random coarse coding comparing with regular-grid approaches, and presents a new approach that combines the natural gradient actor-critic with the random rectangular coarse coding.

UR - http://www.scopus.com/inward/record.url?scp=56749175477&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=56749175477&partnerID=8YFLogxK

U2 - 10.1109/SICE.2008.4654995

DO - 10.1109/SICE.2008.4654995

M3 - Conference contribution

SN - 9784907764296

SP - 2027

EP - 2034

BT - Proceedings of SICE Annual Conference 2008 - International Conference on Instrumentation, Control and Information Technology

ER -