This Article 
 Bibliographic References 
 Add to: 
PCA versus LDA
February 2001 (vol. 23 no. 2)
pp. 228-233

Abstract—In the context of the appearance-based paradigm for object recognition, it is generally believed that algorithms based on LDA (Linear Discriminant Analysis) are superior to those based on PCA (Principal Components Analysis). In this communication, we show that this is not always the case. We present our case first by using intuitively plausible arguments and, then, by showing actual results on a face database. Our overall conclusion is that when the training data set is small, PCA can outperform LDA and, also, that PCA is less sensitive to different training data sets.

[1] P.N. Belhumeur, J. Hespanda, and D. Kriegeman, Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[2] K. Etemad and R. Chellapa, “Discriminant Analysis for Recognition of Human Face Images,” J. Optics of Am. A, vol. 14, no. 8 pp. 1724-1733, 1997.
[3] R.A. Fisher, “The Statistical Utilization of Multiple Measurements,” Annals of Eugenics, vol. 8, pp. 376-386, 1938.
[4] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[5] M. Kirby and L. Sirovich,“Application of Karhunen-Loève procedure for the characterization of human faces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 103-108, Jan. 1990.
[6] A.M. Martínez and R. Benavente, “The AR-face database,” CVC Technical Report 24, June 1998.
[7] A. M. Martínez, “Recognition of Partially Occluded and/or Imprecisely Localized Faces Using a Probabilistic Approach,” Proc. Computer Vision and Pattern Recognition, vol. 1, pp. 712-717, June 2000.
[8] H. Moon and P.J. Phillips, “Analysis of PCA-based Face Recognition Algorithms,” Empirical Evaluation Techniques in Computer Vision, K.J. Bowyer and P.J. Phillips, eds., IEEE CS, 1998.
[9] B. Moghaddam and A. Pentland, “Probabilistic Visual Learning for Object Representation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 696-710, July 1997.
[10] H. Murakami, B.V.K. Vijaya Kumar, “Efficient Calculation of Primary Images from a Set of Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 4, no. 5, pp. 511-515, 1982.
[11] H. Murase, F. Kimura, M. Yoshimura, and Y. Miyake, “An Improvement of the Autocorrelation Matrix in Pattern Matching Method and Its Application to Handprinted‘HIRAGANA’,” Trans. IECE Japan, vol. 64D, no. 3, 1981.
[12] H. Murase and S.K. Nayar, “Visual Learning and Recognition of 3-D Objects from Appearance,” Int'l J. Computer Vision, vol. 14, pp. 5-24, 1995.
[13] S.K. Nayar, N.A. Nene, and H. Murase, “Subspace Methods for Robot Vision,” IEEE Trans. Robotics and Automation, vol. 12, no. 5, pp. 750-758, 1996.
[14] A. Pentland, T. Starner, N. Etcoff, N. Masoiu, O. Oliyide, and M. Turk, “Experiments with Eigenfaces,” Proc. Looking at People Workshop Int'l Joint Conf. Artifical Intelligence, Aug. 1993.
[15] P.J. Phillips, H. Moon, P. Rauss, and S.A. Rizvi, "The FERET Evaluation Methodology for Face-Recognition Algorithms," Computer Vision and Pattern Recognition, pp. 137-143, 1997.
[16] L. Sirovich and M. Kirby, “A Low-Dimensional Procedure for the Characterization of Human Faces,” J. Optical Soc. Am. A, vol. 4, no. 3, pp. 519-524, 1987.
[17] D.L. Swets and J. Weng, Using Discriminant Eigenfeatures for Image Retrieval IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 8, pp. 831-836, Aug. 1996.
[18] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
[19] J.J. Weng, “Crescepton and SHOSLIF: Towards Comprehensive Visual Learning,” Early Visual Learning, S.K. Nayar and T. Poggio, eds., pp. 183-214, Oxford Univ. Press, 1996.

Index Terms:
Face recognition, pattern recognition, principal components analysis, linear discriminant analysis, learning from undersampled distributions, small training data sets.
Aleix M. Martínez, Avinash C. Kak, "PCA versus LDA," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 228-233, Feb. 2001, doi:10.1109/34.908974
Usage of this product signifies your acceptance of the Terms of Use.