• Publication
  • 2013
  • Issue No. 11 - Nov.
  • Abstract - Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets
Nov. 2013 (vol. 35 no. 11)
pp. 2706-2719
J. A. Perez-Carrasco, Dept. Teor. de la Senal y Comun., Univ. of Sevilla, Sevilla, Spain
Bo Zhao, Sch. of Electr. & Electron. Eng., Nanyang Technol. Univ., Singapore, Singapore
C. Serrano, Dept. Teor. de la Senal y Comun., Univ. of Sevilla, Sevilla, Spain
B. Acha, Dept. Teor. de la Senal y Comun., Univ. of Sevilla, Sevilla, Spain
T. Serrano-Gotarredona, Inst. de Microelectron. de Sevilla, Sevilla, Spain
Shouchun Chen, Sch. of Electr. & Electron. Eng., Nanyang Technol. Univ., Singapore, Singapore
B. Linares-Barranco, Inst. de Microelectron. de Sevilla, Sevilla, Spain
Event-driven visual sensors have attracted interest from a number of different research communities. They provide visual information in quite a different way from conventional video systems consisting of sequences of still images rendered at a given "frame rate." Event-driven vision sensors take inspiration from biology. Each pixel sends out an event (spike) when it senses something meaningful is happening, without any notion of a frame. A special type of event-driven sensor is the so-called dynamic vision sensor (DVS) where each pixel computes relative changes of light or "temporal contrast." The sensor output consists of a continuous flow of pixel events that represent the moving objects in the scene. Pixel events become available with microsecond delays with respect to "reality." These events can be processed "as they flow" by a cascade of event (convolution) processors. As a result, input and output event flows are practically coincident in time, and objects can be recognized as soon as the sensor provides enough meaningful events. In this paper, we present a methodology for mapping from a properly trained neural network in a conventional frame-driven representation to an event-driven representation. The method is illustrated by studying event-driven convolutional neural networks (ConvNet) trained to recognize rotating human silhouettes or high speed poker card symbols. The event-driven ConvNet is fed with recordings obtained from a real DVS camera. The event-driven ConvNet is simulated with a dedicated event-driven simulator and consists of a number of event-driven processing modules, the characteristics of which are obtained from individually manufactured hardware modules.
Index Terms:
image representation,cameras,computer vision,feedforward neural nets,image coding,event-driven processing module,frame-driven vision system,frame-free event-driven vision system,low-rate rate coding,coincidence processing,feedforward ConvNets,event-driven visual sensor,visual information,dynamic vision sensor,DVS,temporal contrast,event flow processing,neural network mapping,frame-driven representation,event-driven representation,convolutional neural network,dedicated event-driven simulator,Neurons,Sensors,Voltage control,Visualization,Feature extraction,Neural networks,Dynamic range,high speed vision,Feature extraction,convolutional neural networks,object recognition,spiking neural networks,event-driven neural networks,bioinspired vision
Citation:
J. A. Perez-Carrasco, Bo Zhao, C. Serrano, B. Acha, T. Serrano-Gotarredona, Shouchun Chen, B. Linares-Barranco, "Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNets," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 11, pp. 2706-2719, Nov. 2013, doi:10.1109/TPAMI.2013.71
Usage of this product signifies your acceptance of the Terms of Use.