2011 13th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (2005)
Sept. 25, 2005 to Sept. 29, 2005
Răzvan V. Florian , Center for Cognitive and Neural Studies, University of Genoa and Babeş-Bolyai University
The paper presents a new reinforcement learning mechanism for spiking neural networks. The algorithm is derived for networks of stochastic integrate-and-fire neurons, but it can be also applied to generic spiking neural networks. Learning is achieved by synaptic changes that depend on the firing of pre- and postsynaptic neurons, and that are modulated with a global reinforcement signal. The ef- ficacy of the algorithm is verified in a biologically-inspired experiment, featuring a simulated worm that searches for food. Our model recovers a form of neural plasticity experimentally observed in animals, combining spike-timing-dependent synaptic changes of one sign with nonassociative synaptic changes of the opposite sign determined by presynaptic spikes. The model also predicts that the time constant of spike-timing-dependent synaptic changes is equal to the membrane time constant of the neuron, in agreement with experimental observations in the brain. This study also led to the discovery of a biologically-plausible reinforcement learning mechanism that works by modulating spike-timing-dependent plasticity (STDP) with a global reward signal.
Răzvan V. Florian, "A Reinforcement Learning Algorithm for Spiking Neural Networks", 2011 13th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, vol. 00, no. , pp. 299-306, 2005, doi:10.1109/SYNASC.2005.13