This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Competitive Learning Algorithms and Neurocomputer Architecture
August 1998 (vol. 47 no. 8)
pp. 847-858

Abstract—This paper begins with an overview of several competitive learning algorithms in artificial neural networks, including self-organizing feature maps, focusing on properties of these algorithms important to hardware implementations. We then discuss previously reported digital implementations of these networks. Finally, we report a reconfigurable parallel neurocomputer architecture we have designed using digital signal processing chips and field-programmable gate array devices. Communications are based upon a broadcast network with FPGA-based message preprocessing and postprocessing. A small prototype of this system has been constructed and applied to competitive learning in self-organizing maps. This machine is able to model slowly-varying nonstationary data in real time.

[1] T. Kohonen, "The Self-Organizing Map," Proc. IEEE, vol. 78, pp. 1,464-1,480, Sept. 1990.
[2] DARPA, DARPA Neural Network Study.Fairfax, Va.: AFCEA Int'l Press, 1988.
[3] K.W. Przytula and V.K. Prasanna, Parallel Digital Implementations of Neural Networks.Englewood Cliffs, N.J.: Prentice Hall, 1993.
[4] B.J. Sheu and J. Choi, Neural Information Processing and VLSI.Boston: Kluwer, 1995.
[5] J. Burr, "Digital Neurochip Design," Parallel Digital Implementations of Neural Networks, K.W. Przytula and V.K. Prasanna, eds., chapter 8, pp. 223-281.Englewood Cliffs, N.J.: Prentice Hall, 1993.
[6] S.Y. Kung, Digital Neural Networks.Englewood Cliffs, N.J.: Prentice Hall, 1993.
[7] J.N.H. Heemskerk, "Overview of Neural Hardware," Neurocomputers for Brain Style Processing: Design, Implementation and Application, PhD thesis, chapter 3, Unit of Experimental and Theoretical Psychology, Leiden Univ., The Netherlands, 1995.
[8] D. Hammerstrom, W. Henry, and M. Kuhn, "Neurocomputer System for Neural network Applications," Parallel Digital Implementations of Neural Networks, K.W. Przytula and V.K. Prasanna, eds., pp. 107-138.Englewood Cliffs, N.J.: Prentice Hall, 1993.
[9] H.T. Kung, "Why Systolic Architectures," Computer, vol. 15, pp. 37-46, 1982.
[10] S.Y. Kung and J.N. Hwang, "A Unifying Algorithm/Architecture for Artificial Neural Networks," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 4, pp. 2,505-2,508, 1989.
[11] H.C. Card, "Digital VLSI Backpropagation Networks," Canadian J. Electrical and Computer. Eng., vol. 20, pp. 15-23, Jan. 1995.
[12] J. Bailey and D. Hammerstrom, "Why VLSI Multiplexing of Associative VLCNs Requires Connection Multiplexing," Proc. IEEE Int'l Conf. Neural Networks, vol. 2, pp. 173-180, 1990.
[13] C.E. Cox and E. Blanz, "GANGLION—A Fast Field-Programmable Gate Array Implementation of a Connectionist Classifier," IEEE J. Solid-State Circuits, vol. 27, no. 3, pp. 288-299, Mar. 1992.
[14] S.Y. Kung and J.N. Hwang, "A Unifying Algorithm/Architecture for Artificial Neural Networks," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 4, pp. 2,505-2,508, 1989.
[15] S. Mackie and J.S. Denker, "A Digital Implementation of a Best Match Classifier," IEEE Custom Integrated Circuits Conf., pp. 10.4.1-10.4.4, 1988.
[16] M.S. Melton, T. Phan, D.S. Reeves, and D.E. van der Bout, “The TinMANN VLSI Chip,” IEEE Trans. Neural Networks, vol. 3, pp. 375-384, May 1992.
[17] A. Rao, M.R. Walker, L.T. Clark, L.A. Akers, and R.O. Grondin, "VLSI Implementation of Neural Classifiers," Neural Computation, vol. 2, pp. 35-43, 1990.
[18] H.C. Card, "Analog Circuits for Relaxation Networks," Int'l J. Neural Systems, vol. 4, pp. 359-379, Dec. 1993.
[19] M.A. Jabri, R.J. Coggins, and B.G. Flower, Adaptive Analog VLSI Neural Systems.London: Chapman and Hall, 1996.
[20] S. Grossberg, "Adaptive Pattern Classification and Universal Recoding: I. Parallel Development and Coding of Neural Feature Detectors," Biological Cybernetics, vol. 23, pp. 121-134, 1976.
[21] S. Grossberg, "Competitive Learning: From Interactive Activation to Adaptive Resonance," Cognitive Science, vol. 11, pp. 23-63, 1987.
[22] T. Kohonen, "Self-Organized Formation of Topologically Correct Feature Maps," Biological Cybernetics, vol. 43, pp. 59-69, 1982.
[23] D.E. Rumelhart and D. Zipser, "Feature Discovery by Competitive Learning," Cognitive Science, vol. 9, pp. 75-112, 1985.
[24] J. Hertz, A. Krogh, and R.G. Palmer, Introduction to the Theory of Neural Computation. Addison-Wesley, 1991.
[25] A.K. Krishnamurthy, S.C. Ahalt, D.E. Melton, and P. Chen, "Neural Networks for Vector Quantization of Speech and Images," IEEE J. Selected Areas in Comm., vol. 8, pp. 1,449-1,457, Oct. 1990.
[26] D. DeSieno, "Adding a Conscience to Competitive Learning," Proc IEEE Int'l Conf. Neural Networks, vol. 1, pp. 117-124,San Diego, Calif., 1988.
[27] R.O. Duda and P.E. Hart, Pattern Recognition and Scene Analysis.New York: John Wiley, 1973.
[28] T. Kohonen, "Self-Organization and Associated Memory," Berlin Heidelberg. New York: Springer-Verlag, 1988.
[29] S. Kamarsu and H.C. Card, "Limited-Precision Unsupervised Learning Algorithms for Speech Coding," Proc. INNS World Congress Neural Networks, vol. 1, pp. 128-131,Washington, D.C., July 1995.
[30] M.R. Schroeder and B.S. Atal, "Code-Excited Linear Prediction (CELP); High Quality Speech at Very Low Bit Rates" Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 937-940, 1985.
[31] C. von der Malsburg, "Self Organization of Orientation Selective Cells in Striate Cortex," Kybernetik, vol. 14, pp. 85-100, 1973.
[32] D.J. Willshaw and C. von der Malsburg, "How Patterned Neural Connections Can Be Set Up by Self Organization," Proc. Royal Soc. London, vol. B194, pp. 431-445, 1976.
[33] C. Von der Malsburg, "Network Self-Organization" An Introduction to Neural and Electronic Networks, S.F. Zornetzer, J.L. Davis, and C. Lau, eds.. San Diego, Calif.: Academic Press, 1990.
[34] P. Thiran, V. Peiris, P. Heim, and B. Hochet, "Quantization Effects in Digitally Behaving Circuit Implementations of Kohonen Networks," IEEE Trans. Neural Networks, vol. 5, pp. 450-458, May 1994.
[35] G.A. Carpenter and S. Grossberg, "A Self-Organizing Neural Network for Supervised Learning, Recognition, and Prediction," IEEE Comm., pp. 38-49, Sept 1992.
[36] Y. Hirai, "PDM Digital Neural Network System," Parallel Digital Implementations of Neural Networks, K.W. Przytula and V.K. Prasanna, eds., pp. 283-311,Englewood Cliffs, N.J.: Prentice Hall, 1993.
[37] B.R. Gaines, "Stochastic Computing Systems," Advances in Information Systems Science, J.F. Tou, ed., vol. 2, chapter 2, pp. 37-172.New York: Plenum Press, 1969.
[38] J.E. Tomberg and K. Kaski, “Pulse Density Modulation Technique in VLSI Implementation of Neural Network Algorithms,” IEEE J. Solid-State Circuits, vol. 25, pp. 1277-1286, 1990.
[39] M.S. Tomlinson, D.J. Walker, and M.A. Sivilotti, “A Digital Neural Network Architecture for VLSI,” Proc. Int'l Joint Conf. Neural Networks, vol. 2, pp. 545-550, 1990.
[40] A.F. Murray, D. Del Corso, and L. Tarassenko, “Pulse‐Stream VLSI Neural Networks Mixing Analog and Digital Techniques,” IEEE Trans. Neural Networks, Vol. 2, No. 2, Mar. 1991, pp. 193‐204.
[41] K.A. Grajski, "Neurocomputing Using the MasPar MP-1," Parallel Digital Implementations of Neural Networks, K.W. Przytula and V.K. Prasanna, eds., pp. 51-76.Englewood Cliffs, N.J.: Prentice Hall, 1993.
[42] G.K. Rosendahl, "Polymorphic Computing Paradigms Realized for a FPD Based Multicomputer," PhD thesis, Dept. of Electrical and Computer Eng., Univ. of Manitoba, 1995.
[43] Course Notes for 6.270, The MIT Lego Robot Design Course, Dept. of Electrical Eng. and Computer Science, Massachussetts Inst. of Tech nology, 1994. Contactftp.cs.utexas.edu6.270-organizers@athena.mit.edu .
[44] U. Ramacher et al., "Multiprocessor and Memory Architecture of the Neurocomputer Synapse-1," Proc. MicroNeuro, 1993.
[45] H. Speckmann, P. Thole, M. Bogdan, and W. Rosenstiel, "Coprocessors for Special Neural Networks," Proc Int'l Conf. Neural Netwroks,Orlando, Fla., 1994.
[46] X. Fang, P. Thole, J. Goeppert, and W. Rosenstiel, "A Hardware Supported System for a Special Online Application of Self-Organizing Maps," Proc. Int'l Conf. Neural Networks, vol. II, pp. 956-961, 1996.

Index Terms:
Computer architecture, parallel processing, neurocomputers, field programmable devices, artificial neural networks, competitive learning, self-organizing feature maps.
Citation:
H.c. Card, G.k. Rosendahl, D.k. McNeill, R.d. McLeod, "Competitive Learning Algorithms and Neurocomputer Architecture," IEEE Transactions on Computers, vol. 47, no. 8, pp. 847-858, Aug. 1998, doi:10.1109/12.707586
Usage of this product signifies your acceptance of the Terms of Use.