Subscribe
Issue No.12 - December (2009 vol.31)
pp: 2298-2304
Carlos D. Castillo , University of Maryland, College Park
David W. Jacobs , University of Maryland, College Park
ABSTRACT
Face recognition across pose is a problem of fundamental importance in computer vision. We propose to address this problem by using stereo matching to judge the similarity of two, 2D images of faces seen from different poses. Stereo matching allows for arbitrary, physically valid, continuous correspondences. We show that the stereo matching cost provides a very robust measure of similarity of faces that is insensitive to pose variations. To enable this, we show that, for conditions common in face recognition, the epipolar geometry of face images can be computed using either four or three feature points. We also provide a straightforward adaptation of a stereo matching algorithm to compute the similarity between faces. The proposed approach has been tested on the CMU PIE data set and demonstrates superior performance compared to existing methods in the presence of pose variation. It also shows robustness to lighting variation.
INDEX TERMS
Face recognition, pose, stereo matching, epipolar geometry.
CITATION
Carlos D. Castillo, David W. Jacobs, "Using Stereo Matching with General Epipolar Geometry for 2D Face Recognition across Pose", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 12, pp. 2298-2304, December 2009, doi:10.1109/TPAMI.2009.123
REFERENCES
 [1] Labeled Faces in the Wild Website, http://vis-www.cs.umass.edu/lfwresults.html , 2009. [2] A.B. Ashraf, S. Lucey, and T. Chen, “Learning Patch Correspondences for Improved Viewpoint Invariant Face Recognition,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, June 2008. [3] R. Basri and D. Jacobs, “Lambertian Reflectance and Linear Subspaces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003. [4] D. Beymer and T. Poggio, “Face Recognition from One Example View,” Technical Report AIM-1536, 1995. [5] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3d Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003. [6] K.W. Bowyer, K.I. Chang, and P.J. Flynn, “A Survey of Approaches and Challenges in 3d and Multi-Modal $3{\rm d} + 2{\rm d}$ Face Recognition,” Computer Vision and Image Understanding, vol. 101, no. 1, pp. 1-15, 2006. [7] C.D. Castillo and D.W. Jacobs, “Using Stereo Matching for 2D Face Recognition across Pose,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007. [8] X. Chai, S. Shan, X. Chen, and W. Gao, “Locally Linear Regression for Pose-Invariant Face Recognition,” IEEE Trans. Image Processing, vol. 16, no. 7, pp. 1716-1725, July 2007. [9] I.J. Cox, S.L. Hingorani, S.B. Rao, and B.M. Maggs, “A Maximum Likelihood Stereo Algorithm,” Computer Vision and Image Understanding, vol. 63, no. 3, pp. 542-567, 1996. [10] A. Criminisi, A. Blake, C. Rother, J. Shotton, and P.H.S. Torr, “Efficient Dense Stereo with Occlusions for New View-Synthesis by Four-State Dynamic Programming,” Int'l J. Computer Vision, vol. 71, no. 1, pp. 89-110, 2007. [11] J. Domke and Y. Aloimonos, “A Probabilistic Framework for Correspondence and Egomotion,” Proc. Workshop Dynamical Vision, R. Vidal, A. Heyden, and Y. Ma, eds., pp. 232-242, 2006. [12] J. Domke and Y. Aloimonos, “A Probabilistic Notion of Correspondence and the Epipolar Constraint,” Proc. Third Int'l Symp. 3D Data Processing, Visualization, and Transmission, pp. 41-48, 2006. [13] A.S. Georghiades, P.N. Belhumeur, and D.J. Kriegman, “From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001. [14] Y. Gizatdinova and V. Surakka, “Feature-Based Detection of Facial Landmarks from Neutral and Expressive Facial Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 135-139, Jan. 2006. [15] R. Gross, S. Baker, I. Matthews, and T. Kanade, “Face Recognition across Pose and Illumination,” Handbook of Face Recognition, S.Z. Li and A.K. Jain, eds., Springer-Verlag, June 2004. [16] R. Gross and V. Brajovic, “An Image Preprocessing Algorithm for Illumination Invariant Face Recognition,” Proc. Fourth Int'l Conf. Audio- and Video-Based Biometric Person Authentication, June 2003. [17] R. Gross, I. Matthews, and S. Baker, “Appearance-Based Face Recognition and Light-Fields,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 4, pp. 449-465, Apr. 2004. [18] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004. [19] C.-K. Hsieh and Y.-C. Chen, “Kernel-Based Pose Invariant Face Recognition,” Proc. IEEE Int'l Conf. Multimedia and Expo, pp. 987-990, 2007. [20] G.B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments,” Proc. Faces in Real-Life Images Workshop in European Conf. Computer Vision, 2008. [21] D.G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004. [22] S. Lucey and T. Chen, “A Viewpoint Invariant, Sparsely Registered, Patch Based, Face Verifier,” Int'l J. Computer Vision, vol. 80, pp. 58-71, Oct. 2008. [23] A. Martinez, “Matching Expression Variant Faces,” Vision Research, vol. 9, pp. 1047-1060, 2003. [24] S. Romdhani, V. Blanz, and T. Vetter, “Face Identification by Fitting a 3d Morphable Model Using Linear Shape and Texture Error Functions,” Proc. European Conf. Computer Vision, vol. 4, pp. 3-19, 2002. [25] T. Sim, S. Baker, and M. Bsat, “The cmu Pose, Illumination, and Expression Database,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003. [26] M.A. Turk and A.P. Pentland, “Face Recognition Using Eigenfaces,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 586-591, 1991. [27] A. Vedaldi and B. Fulkerson, “VLFeat: An Open and Portable Library of Computer Vision Algorithms,” http:/www.vlfeat.org/, 2008. [28] P. Viola and M. Jones, “Robust Real-Time Object Detection,” Int'l J. Computer Vision, 2002. [29] L. Wiskott, J.-M. Fellous, N. Krüger, and C. von der Malsburg, “Face Recognition by Elastic Bunch Graph Matching,” Proc. Seventh Int'l Conf. Computer Analysis of Images and Patterns, G. Sommer, K. Daniilidis, and J. Pauli, eds., pp. 456-463, 1997. [30] W. Zhao, R. Chellappa, P.J. Phillips, and A. Rosenfeld, “Face Recognition: A Literature Survey,” ACM Computing Surveys, vol. 35, no. 4, pp. 399-458, Dec. 2003.