This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Stochastic Neural Computation II: Soft Competitive Learning
September 2001 (vol. 50 no. 9)
pp. 906-920

Abstract—An investigation has been made into the use of stochastic arithmetic to implement an artificial neural network solution to a typical pattern recognition application. Optical character recognition is performed on very noisy characters in the E-13B MICR font. The artificial neural network is composed of two layers, the first layer being a set of soft competitive learning subnetworks and the second a set of fully connected linear output neurons. The observed number of clock cycles in the stochastic case represents an order of magnitude improvement over the floating-point implementation assuming clock frequency parity. Network generalization capabilities were also compared based on the network squared error as a function of the amount of noise added to the input patterns. The stochastic network maintains a squared error within 10 percent of that of the floating-point implementation for a wide range of noise levels.

[1] B.D. Brown and H.C. Card, “Stochastic Neural Computation I: Computational Elements,” IEEE Trans. Computers, vol. 50, no. 9, pp. 891-905, Sept. 2001.
[2] J. Hertz, A. Krogh, and R.G. Palmer, Introduction to the Theory of Neural Computation. Addison-Wesley, 1991.
[3] C.M. Bishop, Neural Networks for Pattern Recognition. Clarendon Press, 1995.
[4] B.D. Ripley, Pattern Recognition and Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1996.
[5] J.A. Vlontzos and S.Y. Kung, “Hidden Markov Models for Character Recognition,” IEEE Trans. Image Processing, vol. 1, no. 4, pp. 539-543, 1989.
[6] S.R. Veltman and R. Prasad, “Hidden Markov Models Applied to On-Line Handwritten Isolated Character Recognition,” IEEE Trans. Image Processing, vol. 3, no. 3, pp. 314-318, 1994.
[7] G.C.H. Chuang and C.C.J. Kuo, “A Wavelet Descriptor of Planar Curves: Theory and Applications,” IEEE Trans. Image Processing, vol. 5, pp 56-70, 1996.
[8] K. Fukushima and N. Wake, “Handwritten Alphanumeric Character Recognition by the Neocognitron,” IEEE Trans Neural Networks, vol. 2, pp. 355-365, 1991.
[9] Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, and L.D. Jackel, “Handwritten Digit Recognition with a Backpropagation Network,” Advances in Neural Information Processing Systems 2, D.S. Touretzky, ed., pp. 396-404, San Mateo, Calif.: Morgan Kaufmann, 1990.
[10] L. Guyon, D. Henderson, P. Albrecht, Y. LeCun, and P. Denker, “Writer-Independent and Writer-Adaptive Neural Network for On-Line Character Recognition,” From Pixels to Features III, S. Impedovo, ed., pp. 493-506, Amsterdam: Elsevier, 1992.
[11] S.Y. Kung and J.S. Taur, “Decision Based Neural Networks with Signal/Image Classification Applications,” IEEE Trans. Neural Networks, vol. 6, pp. 170-181, 1995.
[12] D.A. Karras and S.J. Perantonis, “An Efficient Constrained Training Algorithm for Feedforward Networks,” IEEE Trans. Neural Networks, vol. 6, pp. 1420-1434, 1995.
[13] S.B. Cho, “Neural Network Classifiers for Totally Unconstrained Handwritten Numerals,” IEEE Trans. Neural Networks, vol. 8, pp. 43-53, 1997.
[14] F. Smieja, “The Pandemonium System of Reflective Agents,” IEEE Trans. Neural Networks, vol. 7, pp. 97-106, 1996.
[15] M.D. Ganis, C.L. Wilson, and J.L. Blue, “Neural Network-Based Systems for Handprint OCR Applications,” IEEE Trans. Image Processing, vol. 7, pp. 1097-1112, 1998.
[16] L.S. Yaeger, B.J. Webb, and R.F. Lyon, “Combining Neural Networks and Context-Driven Search for On-Line Printed Handwriting Recognition in the Newton,” Neural Networks: Tricks of the Trade, G.B. Orr and K.R. Muller, eds., chapter 13, Berlin: Springer-Verlag, 1998.
[17] L.P. Morns and S.S. Dlay, “The DSFPN, A New Neural Network for Optical Character Recognition,” IEEE Trans. Neural Networks, vol. 10, pp. 1465-1473, 1999.
[18] C. Mead, Analog VLSI and Neural Systems, Addison-Wesley, Reading, Mass., 1989.
[19] E. Sackinger, B.E. Boser, J. Bromley, Y. LeCun, and L.D. Jackel, “Application of the ANNA Neural Network Chip to High-Speed Character Recognition,” IEEE Trans. Neural Networks, vol. 3, pp. 498-505, 1992.
[20] L.P. Morns and S.S. Dlay, “Analog Design of a New Neural Network for Optical Character Recognition,” IEEE Trans. Neural Networks, vol. 10, pp. 951-953, 1999.
[21] E. Sackinger and H.P. Graf, “A Board System for High-Speed Image Analysis and Neural Networks,” IEEE Trans. Neural Networks, vol. 7, pp. 214-221, 1996.
[22] J.M. Cruz and L.O. Chua, “A 16x16 Cellular Neural Network Universal Chip: The First Complete Single-Chip Dynamic Computer Array with Distributed Memory and Grey-Scale Input-Output,” Analog Integrated Circuits and Signal Processing, vol. 15, pp. 227-237, Mar. 1998.
[23] M. Anguita, F.J. Pelayo, E. Ros, D. Palomar, and A. Prieto, “Focal Plane and Multiple Chip Approaches to CNNs,” Analog Integrated Circuits and Signal Processing, vol. 15, pp. 263-275, Mar. 1998.
[24] Pulsed Neural Networks, W. Maass and C.M. Bishop, eds. Cambridge, Mass.: MIT Press, 1999.
[25] A. Mortara and P. Venier, “Analog VLSI Pulsed Networks for Perceptive Processing,” Pulsed Neural Networks, W. Maass and C.M. Bishop, eds., chapter 7. Cambridge, Mass.: MIT Press, 1999.
[26] J.A. Dickson, R.D. McLeod, and H.C. Card, “Stochastic Arithmetic Implementations of Neural Networks with In Situ Learning,” Proc. Int'l Conf. Neural Networks, pp. 711-716, 1993.
[27] Y.C. Kim and M.A. Shanblatt, “Architecture and Statistical Model of a Pulse-Mode Digital Multilayer Neural Network,” IEEE Trans. Neural Networks, vol. 6, pp. 1109-1118, 1995.
[28] S.J. Nowlan, “Soft Competitive Adaptation: Neural Network Algorithms based on Fitting Statistical Mixtures,” PhD thesis, School of Computer Science, Carnegie-Mellon Univ., 1991.
[29] H.C. Card, S. Kamarsu, and D.K. McNeill, “Competitive Learning and Vector Quantization in Digital VLSI Systems,” Neurocomputing, vol. 18, pp. 195-227, 1998.

Index Terms:
Pulsed neural networks, stochastic arithmetic, competitive learning.
Citation:
Bradley D. Brown, Howard C. Card, "Stochastic Neural Computation II: Soft Competitive Learning," IEEE Transactions on Computers, vol. 50, no. 9, pp. 906-920, Sept. 2001, doi:10.1109/12.954506
Usage of this product signifies your acceptance of the Terms of Use.