The Community for Technology Leaders
2011 13th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (2005)
Timisoara, Romania
Sept. 25, 2005 to Sept. 29, 2005
ISBN: 0-7695-2453-2
pp: 299-306
Răzvan V. Florian , Center for Cognitive and Neural Studies, University of Genoa and Babeş-Bolyai University
ABSTRACT
The paper presents a new reinforcement learning mechanism for spiking neural networks. The algorithm is derived for networks of stochastic integrate-and-fire neurons, but it can be also applied to generic spiking neural networks. Learning is achieved by synaptic changes that depend on the firing of pre- and postsynaptic neurons, and that are modulated with a global reinforcement signal. The ef- ficacy of the algorithm is verified in a biologically-inspired experiment, featuring a simulated worm that searches for food. Our model recovers a form of neural plasticity experimentally observed in animals, combining spike-timing-dependent synaptic changes of one sign with nonassociative synaptic changes of the opposite sign determined by presynaptic spikes. The model also predicts that the time constant of spike-timing-dependent synaptic changes is equal to the membrane time constant of the neuron, in agreement with experimental observations in the brain. This study also led to the discovery of a biologically-plausible reinforcement learning mechanism that works by modulating spike-timing-dependent plasticity (STDP) with a global reward signal.
INDEX TERMS
null
CITATION
Răzvan V. Florian, "A Reinforcement Learning Algorithm for Spiking Neural Networks", 2011 13th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, vol. 00, no. , pp. 299-306, 2005, doi:10.1109/SYNASC.2005.13
90 ms
(Ver 3.3 (11022016))