This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
An Analysis on the Performance of Silicon Implementations of Backpropagation Algorithms for Artificial Neural Networks
December 1991 (vol. 40 no. 12)
pp. 1380-1389

The effects of silicon implementation on the backpropagation learning rule in artificial neural systems are examined. The effects on learning performance of limited weight resolution, range limitations, and the steepness of the activation function are considered. A minimum resolution of about 20/22 bits is generally required, but this figure can be reduced to about 14/15 bits by properly choosing the learning parameter eta which attains good performance in presence of limited resolution. This performance can be further improved by using a modified batch backpropagation rule. Theoretical analysis is compared with ad-hoc simulations and results are discussed in detail.

[1] "80170NW electrically trainable analog neural network," data sheet of INTEL, Mar. 1990.
[2] A. F. Murray and A. V. W. Smith, "Asynchronous VLSI neural networks using pulse-stream arithmetic,"IEEE J. Solid State Circuits, vol. 23, no. 3, pp. 688-697, June 1988.
[3] M. Verleysen and P. G. A. Jespers, "An analog VLSI implementation of Hopfield's neural network,"IEEE Micro, vol. 9, no. 6, pp. 46-55, Dec. 1989.
[4] C. A. Mead,Analog VLSI and Neural Systems. Reading, MA: Addison-Wesley, 1989.
[5] D. D. Caviglia, M. Valle, and G. M. Bisio, "Analog VLSI systems for neural adaptive architectures," note of Dep. Biophisical and Electron. Eng., Univ. of Genoa (I).
[6] D. Del Corso, F. Gregoretti, L. M. Reyneri, and C. Pellegrini, "A pulse stream synapsis based on a closed loop shift register," inParallel Architectures and Neural Networks. Singapore, World Scientific, May 1990, pp. 233-243.
[7] C. Jutten, A. Guerin, and J. Herault, "Simulation machine and integrated implementation of neural networks," inLecture Notes in Computer Sciences, Vol. 412. Berlin, Germany: Springer-Verlag pp. 244-266.
[8] P. Treleaven, M. Pacheco, and M. Vellasco, "VLSI architectures for neural networks,"IEEE Micro, vol. 9, no. 6, pp. 8-27, Dec. 1989.
[9] IEEE Comput. Mag., Special issue on Artificial Neural Systems, Mar. 1988.
[10] P. B. Denyer and J. Mavor, "MOST transconductance multipliers for array applications,"IEE Proc., vol. 128, pt. 1, no. 3, June 1981.
[11] P.D. Wasserman,Neural Computing: Theory and Practice, Van Nostrand Reinhold, New York, 1989.
[12] E. Pasero, L. M. Reyneri, and M. Salmeri, "Short and long term memories on silicon," inParallel Architectures and Neural Networks. Singapore: World Scientific, 1990, pp. 267-278.
[13] R. P. Lippman, "An introduction to computing with neural nets,"IEEE ASSP Msg., vol. 4, pp. 4-22, 1987.
[14] A. H. Kramer and A. Sangiovanni-Vincentelli, "Optimisation techniques for neural networks," Memo. UCB/ERL M89/1, Univ. of Berkeley, Jan. 3, 1989.
[15] K. D. Wagner, C. K. Chin, and E. J. McCluskey, "Pseudorandom/testing."IEEE Trans. Comput., vol. C-36, pp. 332-343, Mar. 1987.
[16] L. M. Reyneri and E. Filippi, "Modified backpropagation algorithm for fast learning in neural networks,"Electron. Lett., vol. 26, no. 19, pp. 1564-1566, Sept. 13, 1990.
[17] L. B. Almeida and F. M. Silva, "Acceleration techniques for the back-propagation algorithm," inLecture Notes in Computer Science, Vol. 412. Berlin, Germany: Springer-Verlag, Feb. 1990, pp. 110-119.
[18] E. Filippi, "RETI NEURALI: Influenza delle Imprecisioni e della Topologia," graduation thesis, Dip. Elettronica, Politecnico di Torino, May 1990.

Index Terms:
VLSI; performance; silicon implementations; backpropagation algorithms; artificial neural networks; learning rule; performance; limited weight resolution; range limitations; steepness; activation function; simulations; Si; artificial intelligence; learning systems; neural nets; VLSI.
Citation:
L.M. Reyneri, E. Filippi, "An Analysis on the Performance of Silicon Implementations of Backpropagation Algorithms for Artificial Neural Networks," IEEE Transactions on Computers, vol. 40, no. 12, pp. 1380-1389, Dec. 1991, doi:10.1109/12.106223
Usage of this product signifies your acceptance of the Terms of Use.