This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
The Science of Making ERORS: What Error Tolerance Implies for Capacity in Neural Networks
April 1992 (vol. 4 no. 2)
pp. 135-144

Discusses the development of formal protocols for handling error tolerance which allow a precise determination of the computational gains that may be expected. The error protocols are illustrated in the framework of a densely interconnected neural network architecture (with associative memory the putative application), and rigorous calculations of capacity ar shown. Explicit capacities are also derived for the case of feedforward neural network configurations.

[1] S. S. Venkatesh, "Epsilon capacity of neural networks," inNeural Networks for Computing(J. Denker, Ed.). New York: AIP, 1986.
[2] R. J. McEliece, E. C. Posner, E. R. Rodemich, and S. S. Venkatesh, "The capacity of the Hopfield associative memory,"IEEE Trans. Inform. Theory, vol. IT-33, pp. 461-482, 1987.
[3] J. Komlós and R. Paturi, "Convergence results in an associative memory model,"Neural Networks, vol. 1, no. 3, pp. 239-250, 1988.
[4] J. J. Hopfield, "Neural networks and physical systems with emergent collective computational properties,"Proc. Nat. Acad. Sci., vol. 79, pp. 2554-2558, 1982.
[5] C. M. Newman, "Memory capacity in neural network models: Rigorous lower bounds,"Neural Networks, vol. 1, no. 3, pp. 223-238, 1988.
[6] S. S. Venkatesh and D. Psaltis, "On reliable computation with formal neurons,"IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, pp. 87-91, 1992.
[7] S. S. Venkatesh, "Computation and learning in the context of neural network capacity," inNeural Networks for Perception, H. Wechsler, Ed. New York: Academic, 1991.
[8] H. Chernoff, "A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations,"Ann. Math. Stat., vol. 23, pp. 493-507, 1952.
[9] I. F. Blake and H. Darabian, "Approximations for the probability in the tails of the binomial distribution,"IEEE Trans. Inform. Theory, vol. IT-33, pp. 426-428, 1987.
[10] L. Schläfli,Gesammelte Mathematische Abhandlungen I. Basel, Switzerland: Verlag Birkhäuser, 1950, pp. 209-212.
[11] J. G. Wendel, "A problem in geometric probability,"Math. Scand., vol. 11, pp. 109-111, 1962.
[12] T. M. Cover, "Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition,"IEEE Trans. Electron. Comput., vol. EC-14, pp. 326-334, 1965.
[13] Z. Füredi, "Random polytopes in thed-dimensional cube,"Discrete Comput. Geom., vol. 1, pp. 315-319, 1986.
[14] J. Kahn, J. Komlós, and E. Szemerédi, "On the determinant of random±1 matrices," preprint.
[15] S. S. Venkatesh and P. Baldi, "Programmed interactions in higher-order neural networks: Maximal capacity,"J. Compl., vol. 7, pp. 316-337, 1991.
[16] E. Baum, "On the capabilities of multilayer perceptrons,"J. Complexity, vol. 4, pp. 193-215, Sept. 1988.
[17] R. J. McEliece,The Theory of Information and Coding. Reading, MA: Addison-Wesley, 1977.

Index Terms:
error tolerance; neural networks; formal protocols; error protocols; densely interconnected neural network architecture; associative memory; feedforward neural network configurations; content-addressable storage; fault tolerant computing; neural nets; protocols
Citation:
S.S. Venkatesh, "The Science of Making ERORS: What Error Tolerance Implies for Capacity in Neural Networks," IEEE Transactions on Knowledge and Data Engineering, vol. 4, no. 2, pp. 135-144, April 1992, doi:10.1109/69.134250
Usage of this product signifies your acceptance of the Terms of Use.