Publication 1998 Issue No. 2 - March/April Abstract - Principal Interconnections in Higher Order Hebbian-Type Associative Memories
Principal Interconnections in Higher Order Hebbian-Type Associative Memories
March/April 1998 (vol. 10 no. 2)
pp. 342-345
 ASCII Text x Jung-Hua Wang, "Principal Interconnections in Higher Order Hebbian-Type Associative Memories," IEEE Transactions on Knowledge and Data Engineering, vol. 10, no. 2, pp. 342-345, March/April, 1998.
 BibTex x @article{ 10.1109/69.683763,author = {Jung-Hua Wang},title = {Principal Interconnections in Higher Order Hebbian-Type Associative Memories},journal ={IEEE Transactions on Knowledge and Data Engineering},volume = {10},number = {2},issn = {1041-4347},year = {1998},pages = {342-345},doi = {http://doi.ieeecomputersociety.org/10.1109/69.683763},publisher = {IEEE Computer Society},address = {Los Alamitos, CA, USA},}
 RefWorks Procite/RefMan/Endnote x TY - JOURJO - IEEE Transactions on Knowledge and Data EngineeringTI - Principal Interconnections in Higher Order Hebbian-Type Associative MemoriesIS - 2SN - 1041-4347SP342EP345EPD - 342-345A1 - Jung-Hua Wang, PY - 1998KW - Associative memoryKW - redundant interconnectionsKW - Hebbian ruleKW - convergence probabilityKW - synchronous updatingKW - confidence interval.VL - 10JA - IEEE Transactions on Knowledge and Data EngineeringER -

Abstract—The existence of principal interconnections useful in solving the proliferation problem in higher order Hebbian-type associative memories is introduced. Among all legal interconnections, we prove there exists a subset Tpr that carries more information than the others. Regardless of the network order p, the elements in Tpr are shown to be those interconnections T that fall within the range of

$$\sqrt {m_s} \le \left| T \right| \le 2 \sqrt {m_s},$$

where ms equals the number of stored codewords. Memories that use only Tpr can maintain original generalization performance, using less than 50 percent of the total number of interconnections.

[1] P. Baldi and S. Venkatesh, "Random Interactions in Higher Order Neural Networks," IEEE Trans. Information Theory, vol. 39, pp. 274-283, 1993.
[2] D. Psaltis, C. Park, and J. Hong, "Higher Order Associative Memories and Their Optical Implementations," Neural Networks, vol. 1, pp. 149-163, 1988.
[3] P.C. Chung and T.F. Krile, "Reliability Characteristics of Quadratic Hebbian-Type Associative Memories in Optical and Electronic Network Implementations," IEEE Trans. Neural Networks, vol. 6, no. 2, pp. 357-367, 1995.
[4] Y. Peng, Z. Zong, and E. McCleney, "Relaxing Backpropagation Networks as Associative Memories," Proc. Int'l Conf. Neural Networks, pp. 1,777-1,782,Perth, Australia, 1995.
[5] C. Giles and T. Maxwell, "Learning, Invariance, and Generalization in Higher Order Networks," Applied Optics, vol. 26, pp. 4,972-4,978, 1987.
[6] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning Internal Representations by Error Propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1: Foundations, D.E. Rumelhart and J.L. McClelland et al., eds., chapter 8, pp. 318-362.Cambridge, Mass.: MIT Press, 1986.
[7] S. Wolpert and E. Micheli-Tzanakou, "A Neuromime in VLSI," IEEE Trans. Neural Networks, vol. 7, no. 2, pp. 300-306, 1996.
[8] R. McEliece, E. Posner, E. Rodemich, and S. Venkatesh, "The Capacity of the Hopfield Associative Memory," IEEE Trans. Information Theory, vol. 33, pp. 461-482, 1987.

Index Terms:
Associative memory, redundant interconnections, Hebbian rule, convergence probability, synchronous updating, confidence interval.
Citation:
Jung-Hua Wang, "Principal Interconnections in Higher Order Hebbian-Type Associative Memories," IEEE Transactions on Knowledge and Data Engineering, vol. 10, no. 2, pp. 342-345, March-April 1998, doi:10.1109/69.683763