This Article 
 Bibliographic References 
 Add to: 
Linear Algebra Approach to Neural Associative Memories and Noise performance of Neural Classifiers
December 1991 (vol. 40 no. 12)
pp. 1429-1435

The authors present an analytic evaluation of saturation and noise performance for a large class of associative memories based on matrix operations. The importance of using standard linear algebra techniques for evaluating noise performance of associative memories is emphasized. The authors present a detailed comparative analysis of the correlation matrix memory and the generalized inverse memory construction rules for auto-associative memory and neural classifiers. Analytic results for the noise performance of neural classifiers that can store several prototypes in one class are presented. The analysis indicates that for neural classifiers the simple correlation matrix memory provides better noise performance than the more complex generalized inverse memory.

[1] M. Minsky and S. Papert,Perceptrons: An Introduction to Computational Geometry. Cambridge, MA: M.I.T. Press, 1969.
[2] J. L. McClelland and D. E. Rumelhart, Eds.,Parallel Distributed Processing, vol. 1, 2. Cambridge, MA: M.I.T. Press, 1986.
[3] C. L. Giles, and T. Maxwell, "Learning and generalization in higher order networks,"Appl. Opt., vol. 26, no. 23, 4972, 1987.
[4] T. Maxwell, "Pattern recognition and single layer networks," inEvolution, Learning and Cognition, Y. C. Lee, Ed. World Scientific, 1988, pp. 347-371.
[5] Y. H. Pao,Adaptive Pattern Recognition and Neutral Networks, Reading, MA: Addison-Wesley, 1989.
[6] J. M. Char, V. Cherkassky, H. Wechsler, and G. L. Zimerman, "Distributed and fault-tolerant computation for retrieval tasks,"IEEE Trans. Comput., vol. C-37, no. 4, pp. 484-490, 1988.
[7] V. Cherkassky, N. Vassilas, G. L. Brodt, and H. Wechsler, "Conventional and associative memory approaches to spelling correction, under review.
[8] F. Rosenblatt,Principles of Neurodynamics. New York: Spartan Books, 1962.
[9] J. A. Anderson, "A simple neural network generating an interactive memory,"Math. Biosci., vol. 14, pp. 197-220, 1972.
[10] J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities,"Proc. Nat. Acad. Sci., vol. 79, pp. 2554-2558, 1982.
[11] T. Kohonen,Self-Organization and Associative Memory. Berlin, Germany: Springer-Verlag, 1988, p. 132.
[12] D. Psaltis and S. S. Venkatesh "Information storage in fully connected networks," inEvolution, Learning and Cognition, Y. C. Lee, Ed. World Scientific, 1988, pp. 51-90.
[13] R. P. Lippman, "An introduction to computing with neural nets,"IEEE ASSP Msg., vol. 4, pp. 4-22, 1987.
[14] E. B. Baum, J. Moody, and F. Wilczek, "Internal representations for associative memory,"Biol. Cybern., vol. 59, pp. 217-228, 1988.
[15] G. S. Stiles and D. L. Denq, "On the effect of noise in the Moore-Penrose generalized inverse associative memory,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-7, no. 3, pp. 358-360, 1985.
[16] D. Casasent and B. Telfer, "Key and recollection vector effects on heteroassociative memory performance,"Appl. Opt., vol. 28, no. 2, pp. 272-283, 1989.
[17] J. Komlos, "On the determinant of (0,1) matrices,"Studia Scientarium Mathematicarum Hungarica, vol. 2, pp. 7-21, 1967.
[18] R. Bellman,Introduction to Matrix Analysis. New York: McGraw-Hill, 1960, p. 56.
[19] G. S. Stiles and D. L. Denq, "A quantitative comparison of the performance of the three discrete distributed associative memory models,"IEEE Trans. Comput., vol. C-36, no. 3, pp. 257-263, 1987.
[20] V. Cherkassky, N. Vassilas, and H. Wechsler, "A hierarchical neural model for semantic categorization,"Int. J. Neural Networks Res. Appl., Learned Information, Oxford, to be published.
[21] N. Vassilas, "Performance of neural associative memories for character recognition and associative database retrieval," Ph.D. dissertation, Dep. EE, Univ. of Minnesota, 1990.
[22] G. Golub and C. F. Van Loan,Matrix Computations. Baltimore, MD: The Johns Hopkins University Press, 1983.
[23] T. Cover and P. Hart, "Nearest neighbor pattern classification,"IEEE Trans. Inform. Theory, vol. IT-13, pp. 21-27, 1967.

Index Terms:
linear algebra approach; neural associative memories; noise performance; neural classifiers; analytic evaluation; saturation; comparative analysis; correlation matrix memory; generalized inverse memory construction rules; content-addressable storage; linear algebra; neural nets; performance evaluation.
V. Cherkassky, K. Fassett, N. Vassilas, "Linear Algebra Approach to Neural Associative Memories and Noise performance of Neural Classifiers," IEEE Transactions on Computers, vol. 40, no. 12, pp. 1429-1435, Dec. 1991, doi:10.1109/12.106229
Usage of this product signifies your acceptance of the Terms of Use.