This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Principal Manifolds and Probabilistic Subspaces for Visual Recognition
June 2002 (vol. 24 no. 6)
pp. 780-788

We investigate the use of linear and nonlinear principal manifolds for learning low-dimensional representations for visual recognition. Several leading techniques: Principal Component Analysis (PCA), Independent Component Analysis (ICA), and nonlinear Kernel PCA (KPCA) are examined and tested in a visual recognition experiment using 1,800+ facial images from the "FERET"database. We compare the recognition performance of nearest-neighbor matching with each principal manifold representation to that of a maximum a posteriori (MAP) matching rule using a Bayesian similarity measure derived from dual probabilistic subspaces. The experimental results demonstrate the simplicity, computational economy, and performance superiority of the Bayesian subspace method over principal manifold techniques for visual matching.

[1] H. Attias, “Independent Factor Analysis,” Neural Computation, vol. 11, no. 4, pp. 803-851, May 1999.
[2] M.S. Bartlett, H.M. Lades, and T.J. Sejnowski, “Independent Component Representations for Face Recognition,” Proc. SPIE, Conf. Human Vision and Electronic Imaging III, vol. 2399, pp. 528-539, 1998.
[3] P.N. Belhumeur, J. Hespanda, and D. Kriegeman, Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[4] C.M. Bishop, “Bayesian PCA,” Advances in Neural Information Processing Systems, pp. 482-388, MIT Press, 1999.
[5] C. Bregler and S.M. Omohundro, “Surface Learning with Applications to Lip Reading,” Advances in Neural Information Processing Systems 6, pp. 43-50, 1994.
[6] R. Brunelli and T. Poggio, "Face Recognition: Features vs. Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 10, pp. 1,042-1,053, Oct. 1993.
[7] M. Burl et al., "Automating the Hunt for Volcanos on Venus," Proc. IEEE Conf. Computer Vision and Pattern Recognition,Seattle, June21-23, 1994.
[8] J.-F. Cardoso, “High-Order Contrasts for Independent Component Analysis,” Neural Computation, vol. 11, no. 1, pp. 157-192, 1999.
[9] P. Comon, “Independent Component Analysis, a New Concept?” Signal Processing, vol. 36, no. 3, 1994.
[10] T.F. Cootes and C.J. Taylor, “Active Shape Models: Smart Snakes,” Proc. British Machine Vision Conf., pp. 9-18, 1992.
[11] R. Courant and D. Hilbert, Methods of Mathematical Physiacs, vol. 1.New York: Interscience, 1953.
[12] D. DeMers and G. Cottrell, “Nonlinear Dimensionality Reduction,” Advances in Neural Information Processing Systems 5, 1993.
[13] K. Etemad and R. Chellappa, “Face Recognition Using Discriminant Eigenvectors,” Proc. IEEE Int'l Conf. Acoustic, Speech, and Signal Processing, pp. 2148-2151, 1996.
[14] B.J. Frey and G.E. Hinton, “Variational Learning in Non-Linear Gaussian Belief Networks,” Neural Computation, vol. 11, no. 1, pp. 193-214, 1999.
[15] T. Hastie, “Principal Curves and Surfaces,” PhD thesis, Stanford Univ., 1984.
[16] T. Hastie and W. Stuetzle, “Principal curves,” J. Am. Statistical Assoc., vol. 84, no. 406, pp. 502-516, 1989.
[17] A. Hyvärinen and E. Oja, “A Family of Fixed-Point Algorithms for Independent Component Analysis,” Technical Report A40, Helsinki Univ. of Tech nology, 1996.
[18] I.T. Jolliffe, Principal Component Analysis. New York: Springer-Verlag, 1986.
[19] M.J. Jones and T. Poggio, “Model-Based Matching by Linear Combination of Prototypes,” AI Memo No. 1583, Artificial Intelligence Laboratory, Massachusetts Inst. of Tech nology. Nov. 1996.
[20] C. Jutten and J. Herault, “Blind Separation of Sources, Part I: An Adaptive Algorithm-Based on Neuromimetic Architecture,” Signal Processing, vol. 24, pp. 1-10, 1991.
[21] M.A. Kramer, “Nonlinear Principal Components Analysis Using Autoassociative Neural Networks,” Am. Inst. Chemical Eng. J., vol. 32, no. 2, pp. 233-243, 1991.
[22] M.M. Loè, Probability Theory. Princeton: Van Nostrand, 1955.
[23] S. Makeig, A.J. Bell, T. Jung, and T.J. Sejnowski, “Independent Component Analysis of Electroencephalographic Data,” Advances in Neural Information Processing Systems 8, pp. 145-151, 1996.
[24] E.C. Malthouse, “Some Theoretical Results on Nonlinear Principal Component Analysis,” technical report, Northwestern Univ., 1998.
[25] M.J. McKeown, S. Makeig, T Jung, A.J. Bell, and T.J. Sejnowski, “Analysis of fMRI Data by Blind Separation into Spatial Independent Components,” Human Brain Mapping, vol. 6, pp. 160-188, 1998.
[26] B. Moghaddam, T. Jebara, and A. Pentland, “Bayesian Modeling of Facial Similarity,” Advances in Neural Information Processing Systems 11, pp. 910-916, 1998.
[27] B. Moghaddam, T. Jebara, and A. Pentland, “Efficient MAP/ML Similarity Matching for Face Recognition,” Proc. Int'l Conf. Pattern Recognition, Aug. 1998.
[28] B. Moghaddam, C. Nastar, and A. Pentland, “Bayesian Face Recognition Using Deformable Intensity Surfaces,” Proc. Computer Vision and Pattern Recognition '96, pp. 638-645, 1996.
[29] B. Moghaddam and A. Pentland, "Probabilistic Visual Learning for Object Detection," Int'l Conf. Computer Vision, 1995, pp. 786-793.
[30] B. Moghaddam and A. Pentland, “Probabilistic Visual Learning for Object Representation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 696-710, July 1997.
[31] B. Moghaddam, W. Wahid, and A. Pentland, “Beyond Eigenfaces: Probabalistic Matching for Face Recognition,” Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 30-35, Nara, Japan, Apr. 1998.
[32] H. Murase and S.K. Nayar, "Learning and Recognition of 3D Objects from Appearance," Proc. IEEE Qualitative Vision Workshop, New York, pp. 39-49, 1993.
[33] H. Murase and S.K. Nayar, “Visual Learning and Recognition of 3-D Objects from Appearance,” Int'l J. Computer Vision, vol. 14, pp. 5-24, 1995.
[34] S. Nayar, S. Baker, and H. Murase, "Parametric Feature Detection," CVPR,San Francisco, 1996.
[35] S.K. Nayar, H. Murase, and S.A. Nene, “General Learning Algorithm for Robot Vision,” Neural and Stochastic Methods in Image and Signal Processing, vol. 2304, 1994.
[36] A. Pentland, B. Moghaddam, and Starner, "View-Based and Modular Eigenspaces for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994, pp. 84-91.
[37] A. Pentland and S. Sclaroff, "Closed-Form Solutions for Physically-Based Shape Modeling and Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 715-729, July 1991.
[38] P.J. Phillips, H. Moon, P. Rauss, and S.A. Rizvi, "The FERET Evaluation Methodology for Face-Recognition Algorithms," Computer Vision and Pattern Recognition, pp. 137-143, 1997.
[39] D. Rubin and D. Thayer, “EM Algorithms for ML Factor Analysis,” Psychometrika, vol. 47, no. 1, pp. 69-76, 1982.
[40] B. Schölkopf, A. Smola, and K.-R. Müller, "Nonlinear Component Analysis as a Kernel Eigenvalue Problem," Neural Computation, Vol. 10, 1998, pp. 1299-1319.
[41] S. Sclaroff and A.P. Pentland, Modal Matching for Correspondence and Recognition IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 6, pp. 545-561, 1995.
[42] M. Tipping and C. Bishop, “Probabilistic Principal Component Analysis,” Technical Report NCRG/97/010, Aston Univ., Sept. 1997.
[43] M.E. Tipping and C.M. Bishop, “Mixtures of Probabilistic Principal Component Analysers,” Neural Computation, vol. 11, no. 2, pp. 443-482, 1999.
[44] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J. Cognitive Neuroscience, vol. 3, no. 1, 1991.
[45] R.S. Zemel and G.E. Hinton, “Developing Population Codes by Minimizing Description Length,” Advances in Neural Information Processing Systems, J.D. Cowan, G. Tesauro, and J. Alspector, eds., vol. 6, Morgan Kaufmann, pp. 11-18, 1994.

Index Terms:
Subspace techniques, PCA, ICA, Kernel PCA, Probabilistic PCA, learning, density estimation, face recognition.
Citation:
Baback Moghaddam, "Principal Manifolds and Probabilistic Subspaces for Visual Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp. 780-788, June 2002, doi:10.1109/TPAMI.2002.1008384
Usage of this product signifies your acceptance of the Terms of Use.