This Article 
 Bibliographic References 
 Add to: 
Noise Performance of Linear Associative Memories
July 1994 (vol. 16 no. 7)
pp. 757-765

The performance of two commonly used linear models of associative memories, generalized inverse (GI) and correlation matrix memory (CMM) is studied analytically in the presence of a new type of noise (training noise due to noisy training patterns). Theoretical expressions are determined for the S/N ratio gain of the GI and CMM memories in the auto-associative and hetero-associative modes of operation. It is found that the GI method performance degrades significantly in the presence of training noise while the CMM method is relatively unaffected by it. The theoretical expressions are plotted and compared with the results obtained from Monte Carlo simulations and the two are found to be in excellent agreement.

[1] M. Minsky and S. Papert,Perceptrons: An Introduction to Computational Geometry. Cambridge, MA: MIT Press, 1969.
[2] C. L. Giles and T. Maxwell, "Learning and generalization in higher order networks,"Appl. Opt., vol. 26, no. 23, p. 4972, 1987.
[3] Y. H. Pao,Adaptive Pattern Recognition and Neutral Networks, Reading, MA: Addison-Wesley, 1989.
[4] T. Maxwell, "Pattern recognition and single layer networks," inEvolution, Learning and Cognition, Y. C. Lee, Ed. New York:World Scientific, 1988, pp. 347-371.
[5] V. Cherkassky, V. Vassilas, G. L. Brodt, and H. Wechsler, "Conventional and associative memory approaches to automatic spelling correction,"Eng. Applicat. Artificial Intell., vol. 5, no. 3, pp. 223-237, 1992.
[6] T. Kohonen, "Correlation memory matrix,"IEEE Trans. Comput., vol. C-21, pp. 353-359, Apr. 1972.
[7] T. Kohonen and M. Ruohonen, "Representation of associated data by matrix operator,"IEEE Trans. Comput., vol. C-22, pp. 701-702, July 1973.
[8] K. Nakano, "Associatron-a model of associative memory,"IEEE Trans. Syst., Man, Cybern., vol. SMC-2, pp. 380-388, July 1972.
[9] K. Murakami and T. Aibara, "Construction of a distributed associative memory on the basis of Bayes discriminant rule,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-3, pp. 210-214, Mar. 1981.
[10] T. Kohonen,Self-Organization and Associative Memory. Berlin, Germany: Springer-Verlag, 1988, p. 132.
[11] M. H. Hassoun and A. M. Youssef, "A high performance recording algorithm for Hopfield model associative memories,"Opt. Eng., vol. 27, pp. 46-54, 1989.
[12] B. Telfer and P. Casasent, "Ho-Kashyap optical associative processors,"Appl. Opt., vol. 29, no. 8, pp. 991-1202, Mar. 1990.
[13] G. S. Stiles and D. Denq, "On the effect of noise on the Moore-Penrose inverse associative memory,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-7, pp. 350-360, May 1985.
[14] K. Murakami and T. Aibara, "An improvement on the Moore-Penrose generalized inverse associative memory,"IEEE Trans. Syst., Man, Cybern., vol. SMC-18, pp. 699-706, July 1987.
[15] G. S. Stiles and D. L. Denq, "A quantitative comparison of the performance of the three discrete distributed associative memory models,"IEEE Trans. Comput., vol. C-36, no. 3, pp. 257-263, 1987.
[16] D. Casasent and B. Telfer, "Key and recollection vector effects on hetero-associative memory performance,"Appl. Opt., vol. 28, no. 2, pp. 272-283, Jan. 1989.
[17] V. Cherkassky, K. Fassett, and N. Vassilas, "Linear algebra approach to neural associative memories and noise performance of neural classifiers,"IEEE Trans. Comput., vol. 40, pp. 1429-1435, Dec. 1991.
[18] J. M. Char, V. Cherkassky, H. Wechsler, and G. L. Zimerman, "Distributed and fault-tolerant computation for retrieval tasks,"IEEE Trans. Comput., vol. 37, no. 4, pp. 484-490, 1988.
[19] K. J. Raghunath and V. U. Reddy, "Finite data performance analysis of MVDR beamformer with and without spatial smoothing,"IEEE Trans. Signal Processing, vol. 40, pp. 2726-2736, Nov. 1992.
[20] K. Murakami and T. Aibara, "Least squares associative memory and a theoretical comparison of its performance,"IEEE Trans. Syst., Man, and Cybern., vol. 20, pp. 1230-1234, Sept./Oct. 1989.
[21] P. O. Oliver, "Optimal noise rejection in linear associative memories,"IEEE Trans. Syst., Man, and Cybern., vol. 19, pp. 814-815, Sept./Oct. 1988.
[22] S. W. Zhang, A. G. Constantinides, and L. H. Zou, "Further noise rejection in linear associative memories,"Neural Net., vol. 5, pp. 163-168, 1992.

Index Terms:
content-addressable storage; neural nets; noise; linear associative memories; generalized inverse; linear models; correlation matrix memory; training noise; S/N ratio gain; auto-associative modes; hetero-associative modes; correlation memory; noise performance
K.J. Raghunath, V. Cherkassky, "Noise Performance of Linear Associative Memories," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 7, pp. 757-765, July 1994, doi:10.1109/34.297959
Usage of this product signifies your acceptance of the Terms of Use.