
This Article  
 
Share  
Bibliographic References  
Add to:  
Digg Furl Spurl Blink Simpy Del.icio.us Y!MyWeb  
Search  
 
ASCII Text  x  
Yoshihiko Hamamoto, Shunji Uchimura, Shingo Tomita, "On the Behavior of Artificial Neural Network Classifiers in HighDimensional Spaces," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 5, pp. 571574, May, 1996.  
BibTex  x  
@article{ 10.1109/34.494648, author = {Yoshihiko Hamamoto and Shunji Uchimura and Shingo Tomita}, title = {On the Behavior of Artificial Neural Network Classifiers in HighDimensional Spaces}, journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence}, volume = {18}, number = {5}, issn = {01628828}, year = {1996}, pages = {571574}, doi = {http://doi.ieeecomputersociety.org/10.1109/34.494648}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, }  
RefWorks Procite/RefMan/Endnote  x  
TY  JOUR JO  IEEE Transactions on Pattern Analysis and Machine Intelligence TI  On the Behavior of Artificial Neural Network Classifiers in HighDimensional Spaces IS  5 SN  01628828 SP571 EP574 EPD  571574 A1  Yoshihiko Hamamoto, A1  Shunji Uchimura, A1  Shingo Tomita, PY  1996 KW  Artificial neural networks KW  generalization error KW  dimensionality KW  training sample size KW  peaking phenomenon KW  1NN classifier KW  Parzen classifier. VL  18 JA  IEEE Transactions on Pattern Analysis and Machine Intelligence ER   
Abstract—It is widely believed in the pattern recognition field that when a fixed number of training samples is used to design a classifier, the generalization error of the classifier tends to increase as the number of features gets large. In this paper, we will discuss the generalization error of the artificial neural network (ANN) classifiers in highdimensional spaces, under a practical condition that the ratio of the training sample size to the dimensionality is small. Experimental results show that the generalization error of ANN classifiers seems much less sensitive to the feature size than 1NN, Parzen and quadratic classifiers.
[1] R.P. Lippmann, "An Introduction to Computing with Neural Nets," IEEE Acoustics, Speech, and Signal Processing Magazine, vol. 4, pp. 422, Apr. 1987.
[2] R.P. Lippmann, "Pattern Classification Using Neural Networks," IEEE Communications Magazine, pp. 4764, Nov. 1989.
[3] S.J. Raudys and A.K. Jain, "Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, pp. 252264, 1991.
[4] A.K. Jain and B. Chandrasekaran, "Dimensionality and Sample Size Considerations in Pattern Recognition Practice," Handbook of Statistics 2, P.R. Krishnaiah and L.N. Kanal, eds., pp. 835855. NorthHolland, 1982.
[5] S. Raudys and A.K. Jain, "Small Sample Size Problems in Designing Artificial Neural Networks," Artificial Neural Networks and Statistical Pattern Recognition Old and New Connections, I.K. Sethi and A.K. Jain, eds., pp. 3350. Elsevier Science Publishers, 1991.
[6] S. Raudys, "On Dimensionality, Learning Sample Size and Complexity of Classification Algorithms," Proc. Third Int'l Conf. Pattern Recognition, pp. 166169, 1976.
[7] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[8] L. Kanal and B. Chandrasekaran, "On Dimensionality and Sample Size in Statistical Pattern Recognition," Pattern Recognition, pp. 238255, 1971.
[9] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning Internal Representations by Error Propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1: Foundations, D.E. Rumelhart and J.L. McClelland et al., eds., chapter 8, pp. 318362.Cambridge, Mass.: MIT Press, 1986.
[10] A.K. Jain and M.D. Ramaswami, "Classifier Design with Parzen Windows," Pattern Recognition and Artificial Intelligence, E.S. Gelsema and L.N. Kanal, eds., pp. 211228. Elsevier Science Publishers, 1988.
[11] K. Fukunaga and D.M. Hummels, "Bayes Error Estimation Using Parzen and kNN Procedures," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, pp. 634643, 1987.
[12] P.A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach. Prentice Hall, 1982.
[13] W.Y. Huang and R.P. Lippmann, "Comparisons Between Neural Net and Conventional Classifiers," Proc. IEEE Int'l Conf. Neural Networks, pp. IV485IV493, 1987.
[14] A. Khotanzad and J.H. Lu, "Distortion Invariant Character Recognition by a MultiLayer Perceptron and BackPropagation Learning," Proc. IEEE Int'l Conf. Neural Networks, pp. I625I632, 1988.
[15] J. Van Ness, "On the Dominance of NonParametric Bayes Rule Discriminant Algorithms in High Dimensions," Pattern Recognition, vol. 12, pp. 355368, 1980.
[16] K. Fukunaga and D.M. Hummels, "Bias of Nearest Neighbor Estimates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, pp. 103112, 1987.
[17] K. Fukunaga and R.R. Hayes, "Effects of Sample Size in Classifier Design," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 8, pp. 873885, Aug. 1989.
[18] R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis. John Wiley&Sons, 1973.
[19] A. Khotanzad and J.H. Lu, "Shape and Texture Recognition by a Neural Network," Artificial Neural Networks and Statistical Pattern Recognition Old and New Connections, I.K. Sethi and A.K. Jain, eds., pp. 109131. Elsevier Science Publishers, 1991.
[20] S. Amari, N. Fujita, and S. Shinomoto, "Four Types of Learning Curves," Neural Computation, vol. 4, pp. 605618, 1992.
[21] E.B. Baum, "When Are kNearest Neighbor and Back Propagation Accurate for Feasible Sized Sets of Examples?" Proc. EURASP Workshop Neural Networks, L.B. Almeida and C.J. Wellekens, eds., Delft Univ. of Tech nology, pp. 225, 1990.
[22] D. Cohn and G. Tesauro, "How Tight are the VapnikChervonenkis Bounds?" Neural Computation, vol. 4, pp. 249269, 1992.
[23] S. Raudys, "Why do Multilayer Perceptrons Have Favorable Small Sample Properties?" Pattern Recognition in Practice 4, E.S. Gelsema and L.N. Kanal, eds., pp. 287298. Elsevier Science B.V., 1994.
[24] R.P.W. Duin, "Superlearning Capabilities of Neural Networks," Proc. Eighth Scandinavian Conf. Image Analysis, Tromso, pp. 547554, 1993.
[25] S. Raudys, "On the Effectiveness of Parzen Window Classifier," Informatica, vol. 2, no. 3, pp. 434454, 1991.
[26] S. Raudys, "On Shape of Pattern Error Function. Initializations and Intrinsic Dimensionality in ANN Classifier Design," Informatica, vol. 4, no. 34, pp. 360383, 1993.
[27] S. Raudys, "Unexpected Small Sample Properties of the Linear Perceptrons," Technical report on research activities in LAFORIAIBP, Univ. PARIS VI, LAFORIA 95/17, 1995.
[28] G.F. Hughes, "On the Mean Accuracy of Statistical Pattern Recognizers," IEEE Trans. Information Theory, vol. 14, pp. 5563, 1968.