This Article 
 Bibliographic References 
 Add to: 
Principal Interconnections in Higher Order Hebbian-Type Associative Memories
March/April 1998 (vol. 10 no. 2)
pp. 342-345

Abstract—The existence of principal interconnections useful in solving the proliferation problem in higher order Hebbian-type associative memories is introduced. Among all legal interconnections, we prove there exists a subset Tpr that carries more information than the others. Regardless of the network order p, the elements in Tpr are shown to be those interconnections T that fall within the range of

$$\sqrt {m_s} \le \left| T \right| \le 2 \sqrt {m_s},$$

where ms equals the number of stored codewords. Memories that use only Tpr can maintain original generalization performance, using less than 50 percent of the total number of interconnections.

[1] P. Baldi and S. Venkatesh, "Random Interactions in Higher Order Neural Networks," IEEE Trans. Information Theory, vol. 39, pp. 274-283, 1993.
[2] D. Psaltis, C. Park, and J. Hong, "Higher Order Associative Memories and Their Optical Implementations," Neural Networks, vol. 1, pp. 149-163, 1988.
[3] P.C. Chung and T.F. Krile, "Reliability Characteristics of Quadratic Hebbian-Type Associative Memories in Optical and Electronic Network Implementations," IEEE Trans. Neural Networks, vol. 6, no. 2, pp. 357-367, 1995.
[4] Y. Peng, Z. Zong, and E. McCleney, "Relaxing Backpropagation Networks as Associative Memories," Proc. Int'l Conf. Neural Networks, pp. 1,777-1,782,Perth, Australia, 1995.
[5] C. Giles and T. Maxwell, "Learning, Invariance, and Generalization in Higher Order Networks," Applied Optics, vol. 26, pp. 4,972-4,978, 1987.
[6] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning Internal Representations by Error Propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1: Foundations, D.E. Rumelhart and J.L. McClelland et al., eds., chapter 8, pp. 318-362.Cambridge, Mass.: MIT Press, 1986.
[7] S. Wolpert and E. Micheli-Tzanakou, "A Neuromime in VLSI," IEEE Trans. Neural Networks, vol. 7, no. 2, pp. 300-306, 1996.
[8] R. McEliece, E. Posner, E. Rodemich, and S. Venkatesh, "The Capacity of the Hopfield Associative Memory," IEEE Trans. Information Theory, vol. 33, pp. 461-482, 1987.

Index Terms:
Associative memory, redundant interconnections, Hebbian rule, convergence probability, synchronous updating, confidence interval.
Jung-Hua Wang, "Principal Interconnections in Higher Order Hebbian-Type Associative Memories," IEEE Transactions on Knowledge and Data Engineering, vol. 10, no. 2, pp. 342-345, March-April 1998, doi:10.1109/69.683763
Usage of this product signifies your acceptance of the Terms of Use.