This Article 
 Bibliographic References 
 Add to: 
On a Constant-Time, Low-Complexity Winner-Take-All Neural Network
April 1995 (vol. 44 no. 4)
pp. 601-604

Abstract—A nearly cost-optimal winner-take-all (WTA) neural network derived from a constant-time sorting network is presented. The resultant WTA network has connection complexity ${\rm O}({\rm n}^{2^{\rm s}/(2^{\rm s} - 1)})$ where s is the depth of cascaded sorting networks. Application of the WTA network to other problems such as nonbinary majority is also included.

[1] T. Kohenon,“Self-organized formation of topologically correct feature maps,” Biological Cybernetics, vol. 43, pp. 59-69, 1982.
[2] D.E. Rumelhart and D. Zipser,“Feature discovery of competitive learning,” Parallel Distributed Processing: Explorations in the Microstructure ofCognition, D.E. Rumelhart and PDP research group, MIT Press, Cambridge, Mass., 1986.
[3] S. Grossberg,“Nonlinear neural networks: Principles, mechanisms, and architecture,” Neural Networks, vol. 1, pp. 17-61, 1988.
[4] B.W. Suter and K.D. Reilly,“On the performance of maximum-picking neural networks,” Int’l J. Neural Networks, vol. 3, no. 3, pp. 85-96, Sept. 1992.
[5] B.W. Suter and M. Kabrisky,“On a magnitude preserving iterative MAXnet algorithm,” Neural Computation, no. 4 pp. 224-233, 1992.
[6] R.P. Lippmann, "An Introduction to Computing with Neural Nets," IEEE Acoustics, Speech, and Signal Processing Magazine, vol. 4, pp. 4-22, Apr. 1987.
[7] G.R. Gindi,A.F. Gmitro,, and K. Parthasarathy,“Winner-take-all networks and associative memory: Analysis and optical implementation,” Proc. Int’l Conf. Neural Networks, vol. III, pp. 607-614,San Diego, Calif., 1987.
[8] J. Lazzaro et al., “Winner-Take-All Networks of O(n) Complexity,” in Advances in Neural Information Processing Systems, Vol. 1,D.S. Touretzky, ed., Morgan Kaufmann, San Francisco, 1989, pp. 703-711.
[9] N.J. Nilsson,Learning Machines: Foundations of Trainable Pattern-Classifying Systems, McGraw-Hill, 1965.
[10] R.J. Williams,“The logic of activation functions,” Parallel Distributed Processing, D.E. Rumelhart and PDP research group, MIT Press, Cambridge, Mass., 1986.
[11] Y.-H. Tseng and J.-L. Wu,“Solving sorting and related problems by quadratic perceptrons,” Electronics Letters, vol. 28, no. 10, pp. 906-908, 1992.
[12] Y. Takefuji and K.-C. Lee,“A super-parallel sorting algorithm based on neural networks,” IEEE Trans. Circuits and Systems, vol. 37, no. 11, pp. 1425-1429, Nov. 1990.
[13] A.K. Chandra,L. Stockmeyer,, and U. Vishkin,“Constant depth reducibility,” SIAM J. of Computing, vol. 13, no. 2, pp. 423-439, May 1984.
[14] K.Y. Siu and J. Bruck, “Neural Computation of Arithmetic Functions,” Proc. IEEE, vol. 78, no. 10, pp. 1,669-1,675, Oct. 1990.
[15] K.-Y. Siu and J. Bruck,“On the power of threshold circuits with smallweights,” SIAM J. of Discrete Math., vol. 4, no. 3, pp. 425-435, Aug. 1991.
[16] S.G. Akl, The Design and Analysis of Parallel Algorithms. Orlando, Fl.: Academic Press, 1989.
[17] B. Widrow and R. G. Winter,“Neural Nets for Adaptive Filtering and Adaptive Pattern Recognition,” IEEE Computer, vol. 21, no. 3, pp. 25-39, March 1988.
[18] R.E. Blahut,Theory and Practice of Error Control Codes, Addison-Wesley, 1983.
[19] Y.-H. Tseng,J. Shiu,, and J.-L. Wu,“Perceptron-based neural networks fordecoding error-correcting codes,” IEEE Int’l Workshop Intelligent SignalProcessing and Comm. Systems,Taipei, pp. 11-22, 1992.
[20] W.J. Wolfe,D. Mathis,C. Anderson,J. Rothman,M. Gottler,G. Brady,R. Walker,G. Duane,, and G. Alaghband,“K-winner networks,” IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 310-315, 1992.
[21] E. Majani,R. Erlanson,, and Y. Abu-Mostafa,“On the k-winner-take-all networks,” Advances in Neural Information Processing Systems I, D.S. Touretzky, ed., pp. 634-624,Los Altos, Calif., Morgan Kaufmann, 1989.

Index Terms:
Quadratic perceptron, sorting, winner-take-all, nonbinary majority, complexity.
Ja-Ling Wu, Yuen-Hsien Tseng, "On a Constant-Time, Low-Complexity Winner-Take-All Neural Network," IEEE Transactions on Computers, vol. 44, no. 4, pp. 601-604, April 1995, doi:10.1109/12.376175
Usage of this product signifies your acceptance of the Terms of Use.