Media type: E-Article Title: Heuristic dynamic programming using echo state network as online trainable adaptive critic Contributor: Koprinkova‐Hristova, Petia; Oubbati, Mohamed; Palm, Günther imprint: Wiley, 2013 Published in: International Journal of Adaptive Control and Signal Processing Language: English DOI: 10.1002/acs.2364 ISSN: 0890-6327; 1099-1115 Keywords: Electrical and Electronic Engineering ; Signal Processing ; Control and Systems Engineering Origination: Footnote: Description: <jats:title>SUMMARY</jats:title><jats:p>The present paper proposes an implementation of a relatively new recurrent neural network architecture—the echo state network (ESN)–within the frame of heuristic dynamic programming. The ESN is trained online to estimate the utility function and to adapt the control policy of an embodied agent. With the advantage of an easy training algorithm, the ESN architecture offers a simple way to calculate the derivatives required for adapting the controller. Experimental results are provided to validate the proposed learning approach. Copyright © 2012 John Wiley & Sons, Ltd.</jats:p>