
This Article  
 
Share  
Bibliographic References  
Add to:  
Digg Furl Spurl Blink Simpy Del.icio.us Y!MyWeb  
Search  
 
ASCII Text  x  
K.Y. Siu, V.P. Roychowdhury, T. Kailath, "DepthSize Tradeoffs for Neural Computation," IEEE Transactions on Computers, vol. 40, no. 12, pp. 14021412, December, 1991.  
BibTex  x  
@article{ 10.1109/12.106225, author = {K.Y. Siu and V.P. Roychowdhury and T. Kailath}, title = {DepthSize Tradeoffs for Neural Computation}, journal ={IEEE Transactions on Computers}, volume = {40}, number = {12}, issn = {00189340}, year = {1991}, pages = {14021412}, doi = {http://doi.ieeecomputersociety.org/10.1109/12.106225}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, }  
RefWorks Procite/RefMan/Endnote  x  
TY  JOUR JO  IEEE Transactions on Computers TI  DepthSize Tradeoffs for Neural Computation IS  12 SN  00189340 SP1402 EP1412 EPD  14021412 A1  K.Y. Siu, A1  V.P. Roychowdhury, A1  T. Kailath, PY  1991 KW  neural computation; parallel computation; size; threshold gates; symmetric Boolean functions; arithmetic functions; depth3; Boolean functions; neural nets; threshold logic. VL  40 JA  IEEE Transactions on Computers ER   
The tradeoffs between the depth (i.e., the time for parallel computation) and the size (i.e., the number of threshold gates) in neural networks are studied. The authors focus the study on the neural computations of symmetric Boolean functions and some arithmetic functions. It is shown that a significant reduction in the size is possible for symmetric functions and some arithmetic functions, at the expense of a small constant increase in depth. In the process, several neural networks which have the minimum size among all the known constructions have been developed. Results on implementing symmetric functions can be used to improve results about arbitrary Boolean functions. In particular, it is shown that any Boolean function can be computed in a depth3 neural network with O(2/sup n/ /sup 2/) threshold gates; it is also proven that at least Omega (2/sup n/ /sup 3/) threshold gates are required.
[1] P. W. Beame, S. A. Cook, and H. J. Hoover, "Log depth circuits for division and related problems,"SIAM J. Comput., vol. 15, pp. 9941003, 1986.
[2] J. Bruck, "Harmonic analysis of polynomial threshold functions,"SIAM J. Discrete Math., May 1990.
[3] J. Bruck and R. Smolensky, "Polynomial threshold functions,AC0functions and spectral norms," Tech. Rep. RJ 7140, IBM Research, Nov. 1989. Appeared inProc. IEEE Symp. Foundations Comput. Sci. Oct. 1990.
[4] A. K. Chandra, L. Stockmeyer, and R. Lipton, "Unbounded fanin circuits and associative functions," inProc. 15th ACM Symp. Theory Comput., 1983, pp. 5260.
[5] A. K. Chandra, L. Stockmeyer, and U. Vishkin, "Constant depth reducibility,"SIAM J. Comput., vol. 13, pp. 423439, 1984.
[6] F. Fich, "Two problems in concrete complexity: Cycle detection and parallel prefix computation," Ph.D. dissertation, Univ. of California, Berkeley, 1982.
[7] M. Furst, J. B. Saxe, and M. Sipser, "Parity, circuits and the polynomialtime hierarchy," inProc. IEEE Symp. Foundations Comput. Sci., vol. 22, 1981, pp. 260270.
[8] A. Hajnal, W. Maass, P. Pudlak, M. Szegedy, and G. Turan, "Threshold circuits of bounded depth,"IEEE Symp. Foundations Comput. Sci., vol. 28, 1987, pp. 99110.
[9] T. Hofmeister, W. Hohberg, and S. Kohling, "Some notes on threshold circuits, and multiplication in depth 4," unpublished manuscript, 1990.
[10] W. Kautz, "The realization of symmetric switching functions with linearinput logical elements,"IRE Trans. Electron. Comput., vol. EC10, 1961.
[11] R. E. Ladner and M. J. Fischer, "Parallel prefix computation,"J. ACM, vol. 27, no. 4, pp. 831838, Oct. 1980.
[12] N. Linial, Y. Mansour, and N. Nisan, "Constant depth circuits, Fourier transforms, and learnability," inProc. 30th IEEE Symp. Foundations Comput. Sci., 1989.
[13] R. Minnick, "Linearinput logic,"IEEE Trans. Electron. Comput., vol. EC10, 1961.
[14] D. E. Muller and F. P. Preparata, "Bounds to complexities of networks for sorting and switching,"J. ACM, vol. 22, no. 2, Apr. 1975.
[15] S. Muroga, "The principle of majority decision logic elements and the complexity of their circuits," inProc. Int. Conf. Inform. Processing, Paris, France, June 1959.
[16] R. Paturi and M. Saks, "On threshold circuits for parity," inProc. IEEE Symp. Foundations Comput. Sci., Oct. 1990.
[17] N. Pippenger, "The complexity of computations by networks,"IBM J. Res. Develop., vol. 31, no. 2, Mar. 1987.
[18] N. P. Redkin, "Synthesis of threshold circuits for certain classes of Boolean functions,"Kibernetika, no. 5, pp. 69, 1970.
[19] J. Reif, "On threshold circuits and polynomial computation," inProc. 2nd Structure in Complexity Theory Conf., 1987, pp. 118123.
[20] V. P. Roychowdhury, K. Y. Siu, A. Orlitsky, and T. Kailath, "On the circuit complexity of neural networks," inAdvances in Neural Information Processing Systems 3, D. S. Touretzky and R. Lippman Eds., Apr. 1991.
[21] V. Roychowdhury, K. Y. Siu, A. Orlitsky, and T. Kailath, "A geometric approach to threshold circuit complexity," inProc. 4th Annu. Workshop Computational Learning Theory (COLT), Aug. 1991.
[22] C. Shannon, "The synthesis of twoterminal switching circuits,"Bell Syst. Tech. J.,vol. 28, pp. 5998, 1949.
[23] K. Y. Siu and J. Bruck, "On the power of threshold circuits with small weights,"SIAM J. Disc. Math., Aug. 1991.
[24] K. Y. Siu and J. Bruck, "Neural computation of arithmetic functions,"Proc. IEEE, vol. 78, pp. 16691675, Oct. 1990.
[25] K. Y. Siu and J. Bruck, "On the dynamic range of linear threshold elements," Tech. Rep. RJ 7237, IBM Research, Jan. 1990.
[26] K. Y. Siu, J. Bruck, and T. Kailath, "Depthefficient neural networks for division and related problems,"IEEE Trans. Inform. Theory, submitted for publication.
[27] K. Y. Siu, V. P. Roychowdhury, and T. Kailath, "Computing with almost optimal size threshold circuits," inProc. Int. Symp. Inform. Theory, Budapest, Hungary, June 1991.
[28] L. Stockmeyer, "On the combinational complexity of certain symmetric Boolean functions,"Math. Syst. Theory, vol. 10, pp. 323336, 1977.
[29] I. Wegener,The Complexity of Boolean Functions. New York: Wiley, 1987.