Readouts for echo-state networks built using locally regularized orthogonal forward regression

Ján Dolinský, Kei Hirose, Sadanori Konishi

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

Echo state network (ESN) is viewed as a temporal expansion which naturally give rise to regressors of various relevance to a teacher output. We illustrate that often only a certain amount of the generated echo-regressors effectively explain the teacher output and we propose to determine the importance of the echo-regressors by a joint calculation of the individual variance contributions and Bayesian relevance using the locally regularized orthogonal forward regression (LROFR). This information can be advantageously used in a variety of ways for an analysis of an ESN structure. We present a locally regularized linear readout built using LROFR. The readout may have a smaller dimensionality than the ESN model itself, and improves robustness and accuracy of an ESN. Its main advantage is ability to determine what type of an additional readout is suitable for a task at hand. Comparison with PCA is provided too. We also propose a radial basis function (RBF) readout built using LROFR, since flexibility of the linear readout has limitations and might be insufficient for complex tasks. Its excellent generalization abilities make it a viable alternative to feed-forward neural networks or relevance-vector-machines. For cases where more temporal capacity is required we propose well studied delay&sum readout.

Original languageEnglish
Pages (from-to)740-762
Number of pages23
JournalJournal of Applied Statistics
Volume45
Issue number4
DOIs
Publication statusPublished - Mar 12 2018

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Readouts for echo-state networks built using locally regularized orthogonal forward regression'. Together they form a unique fingerprint.

  • Cite this