| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
On Dimensionality, Sample Size, and Classification Error of Nonparametric Linear Classification Algorithms
June 1997 (vol. 19 no. 6)
pp. 667-671
Abstract—This paper compares two nonparametric linear classification algorithms—the zero empirical error classifier and the maximum margin classifier—with parametric linear classifiers designed to classify multivariate Gaussian populations [[7]]. Formulae and a table for the mean expected probability of misclassification MEPN are presented. They show that the classification error is mainly determined by N / p, a learning-set size/dimensionality ratio. However, the influences of learning-set size on the generalization error of parametric and nonparametric linear classifiers are quite different. Under certain conditions the nonparametric approach allows us to obtain reliable rules, even in cases where the number of features is larger than the number of training vectors.
[1] S. Amari and N. Murata, "Statistical Theory of Learning Curves Under Entropic Loss Criterion," Neural Computation, vol. 5, pp. 140-153, 1993.
[2] C. Cortes and V. Vapnik, "Support-Vector Networks," Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.
[3] G. McLachlan, Discriminant Analysis and Statistical Pattern Recognition. Wiley, 1992.
[4] R. Meir, "Empirical Risk Minimization Versus Maximum-Likelihood Estimation: A Case Study," Proc. 12th ICPR, vol. 2, Jerusalem, Oct. 1994.
[5] S. Raudys, "Generalization Errors of Adaptive Linear Classifiers," Technical Report LAFORIA 95/17, Institut Blaise Pascal, Univ. Paris VI, May 1995.
[6] S. Raudys, "Linear Classifiers in Perceptron Design," Proc. 13th ICPR, Track D, Wien, Aug. 1996.
[7] S. Raudys and V. Pikelis, "On Dimensionality, Sample Size, Classification Error, and Complexity of Classification Algorithm in Pattern Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 2, no. 3, pp. 242-252, 1980.
[8] H.S. Seung, H. Sompolinsky, and N. Tishby, "Statistical Mechanics From Examples," Physical Review, A, vol. 45, no. 8, pp. 6,056-6,091, 1992.
[9] V.N. Vapnik, Estimation of Dependencies Based on Empirical Data. p. 448 Springer, 1982,.
[10] F.D. Wyman, D. Young, and D. Turner, "A Comparison of Asymptotic Error Rate Expansions for the Sample Linear Discriminant Function," Pattern Recognition, vol. 23, no. 7, pp. 775-783, 1990.
Index Terms:
Generalization error, dimensionality, complexity, sample size, training, margin.
Citation:
Sarunas Raudys, "On Dimensionality, Sample Size, and Classification Error of Nonparametric Linear Classification Algorithms," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 6, pp. 667-671, June 1997, doi:10.1109/34.601254
