This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Appearance-Based Face Recognition and Light-Fields
April 2004 (vol. 26 no. 4)
pp. 449-465

Abstract—Arguably the most important decision to be made when developing an object recognition algorithm is selecting the scene measurements or features on which to base the algorithm. In appearance-based object recognition, the features are chosen to be the pixel intensity values in an image of the object. These pixel intensities correspond directly to the radiance of light emitted from the object along certain rays in space. The set of all such radiance values over all possible rays is known as the plenoptic function or light-field. In this paper, we develop a theory of appearance-based object recognition from light-fields. This theory leads directly to an algorithm for face recognition across pose that uses as many images of the face as are available, from one upwards. All of the pixels, whichever image they come from, are treated equally and used to estimate the (eigen) light-field of the object. The eigen light-field is then used as the set of features on which to base recognition, analogously to how the pixel intensities are used in appearance-based face and object recognition.

[1] E.H. Adelson and J. Bergen, The Plenoptic Function and Elements of Early Vision Computational Models of Visual Processing, Landy and Movshon, eds., MIT Press, 1991.
[2] S. Baker, T. Sim, and T. Kanade, When Is the Shape of a Scene Unique Given Its Light-Field: A Fundamental Theorem of 3D Vision? IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 1, pp. 100-109, Jan. 2003.
[3] P.N. Belhumeur, J. Hespanda, and D. Kriegeman, Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[4] P.N. Belhumeur and D.W. Jacobs, Comparing Images under Variable Illumination Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 610-617, 1998.
[5] P.N. Belhumeur and D.J. Kriegman, What Is the Set of Images of an Object under All Possible Lighting Conditions? Int'l J. Computer Vision, vol. 28, no. 3, pp. 1-16, 1998.
[6] D. Beymer and T. Poggio, “Face Recognition from One Example View,” Proc. Int'l Conf. Computer Vision, pp. 500-507, 1995.
[7] M. Black and A. Jepson, Eigen-Tracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation Int'l J. Computer Vision, vol. 36, no. 2, pp. 101-130, 1998.
[8] D.M. Blackburn, M. Bone, and P.J. Phillips, Facial Recognition Vendor Test 2000: Evaluation Report 2000.
[9] V. Blanz, S. Romdhani, and T. Vetter, Face Identification across Different Poses and Illuminations with a 3D Morphable Model Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 202-207, 2002.
[10] T. Cootes, G. Edwards, and C. Taylor, Active Appearance Models IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-685, June 2001.
[11] F. De La Torre and M. Black, A Framework for Robust Subspace Learning Int'l J. Computer Vision, vol. 54, no. 1, pp. 117-142, 2003.
[12] A.S. Georghiades, P.N. Belhumeur, and D.J. Kriegman, From Few to Many: Generative Models for Recognition under Variable Pose and Illumination Proc. Fourth IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2000.
[13] S.J. Gortler, R. Grzeszczuk, R. Szeliski, and M.F. Cohen, The Lumigraph Computer Graphics Proc. , Ann. Conf. Series (SIGGRAPH), pp. 43-54, 1996.
[14] R. Gross, I. Matthews, and S. Baker, Eigen Light-Fields and Face Recognition across Pose Proc. Fifth IEEE Int'l Conf. Face and Gesture Recognition, 2002.
[15] R. Gross, I. Matthews, and S. Baker, Fisher Light-Fields for Face Recognition Across Pose and Illumination Proc. German Symp. Pattern Recognition, pp. 481-489, 2002.
[16] A. Leonardis and H. Bischof, Robust Recognition Using Eigenimages Computer Vision and Image Understanding, vol. 78, no. 1, pp. 99-118, 2000.
[17] M. Levoy and M. Hanrahan, Light Field Rendering Computer Graphics Proc., Ann. Conf. Series (SIGGRAPH), pp. 31-41, 1996.
[18] I. Matthews and S. Baker, Active Appearance Models Revisited Technical Report CMU-RI-TR-03-02, Carnegie Mellon Univ. Robotics Inst., Apr. 2003.
[19] H. Murase and S.K. Nayar, Visual Learning and Recognition of 3-D Objects from Appearance Int'l J. Computer Vision, vol. 14, pp. 5-24, 1995.
[20] S.G. Narasimhan and S.K. Nayar, Chromatic Framework for Vision in Bad Weather Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000.
[21] A. Pentland, B. Moghaddam, and Starner, "View-Based and Modular Eigenspaces for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994, pp. 84-91.
[22] P.J. Phillips, H. Moon, and S.A. Rozvi, The FERET Evaluation Methodolody for Face Recognition Algorithms IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090-1104, Oct. 2000.
[23] T. Riklin-Raviv and A. Shashua, “The Quotient Image: Class Based Recognition and Synthesis under Varying Illumination,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 566-571, 1999.
[24] S. Romdhani, V. Blanz, and T. Vetter, Face Identification by Matching a 3D Morphable Model Using Linear Shape and Texture Error Functions Proc. European Conf. Computer Vision, pp. 3-19, 2002.
[25] H. Shum, K. Ikeuchi, and R. Reddy, Principal Component Analysis with Missing Data and Its Applications to Polyhedral Object Modeling IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 9, pp. 854-867, Sept. 1995.
[26] T. Sim, S. Baker, and M. Bsat, The CMU Pose, Illumination, and Expression Database IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[27] L. Sirovich and M. Kirby, Low-Dimensional Procedure for the Characterization of Human Faces J. Optical Soc. Am., vol. 4, no. 3, pp. 519-524, 1987.
[28] M. Turk and A. Pentland, "Face Recognition Using Eigenfaces," Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 1991, pp. 586-591.
[29] M.A.O. Vasilescu and D. Terzopoulos, Multilinear Image Analysis for Face Recognition Proc. 16th Int'l Conf. Pattern Recognition, pp. 511-514, 2002.
[30] T. Vetter and T. Poggio, "Linear Object Classes and Image Synthesis from Single Example Image," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 733-741, July 1997.

Index Terms:
Appearance-based object recognition, face recognition, light-fields, eigen light-fields, face recognition across pose.
Citation:
Ralph Gross, Iain Matthews, Simon Baker, "Appearance-Based Face Recognition and Light-Fields," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 4, pp. 449-465, April 2004, doi:10.1109/TPAMI.2004.1265861
Usage of this product signifies your acceptance of the Terms of Use.