The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2013 vol.35)
pp: 624-638
E. Sangineto , Pattern Anal. & Comput. Vision (PAVIS), Ist. Italiano di Tecnol., Genoa, Italy
ABSTRACT
We present an approach to automatic localization of facial feature points which deals with pose, expression, and identity variations combining 3D shape models with local image patch classification. The latter is performed by means of densely extracted SURF-like features, which we call DU-SURF, while the former is based on a multiclass version of the Hausdorff distance to address local classification errors and nonvisible points. The final system is able to localize facial points in real-world scenarios, dealing with out of plane head rotations, expression changes, and different lighting conditions. Extensive experimentation with the proposed method has been carried out showing the superiority of our approach with respect to other state-of-the-art systems. Finally, DU-SURF features have been compared with other modern features and we experimentally demonstrate their competitive classification accuracy and computational efficiency.
INDEX TERMS
Face, Shape, Three dimensional displays, Vectors, Feature extraction, Detectors, efficient feature extraction, Facial feature detection, head pose estimation, Hausdorff distance
CITATION
E. Sangineto, "Pose and Expression Independent Facial Landmark Localization Using Dense-SURF and the Hausdorff Distance", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 3, pp. 624-638, March 2013, doi:10.1109/TPAMI.2012.87
REFERENCES
[1] W. Zhao, R. Chellappa, P.J. Phillips, and A. Rosenfeld, "Face Recognition: A Literature Survey," ACM Computing Surveys, vol. 35, no. 4, pp. 399-458, 2003.
[2] A.M. Martinez, "Recognizing Imprecisely Localized, Partially Occluded and Expression Variant Faces from a Single Sample per Class," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp. 748-763, June 2002.
[3] E. Murphy-Chutorian and M.M. Trivedi, "Head Pose Estimation in Computer Vision: A Survey," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 4, pp. 607-626, Apr. 2009.
[4] R. Fergus, P. Perona, and A. Zisserman, "Object Class Recognition by Unsupervised Scale-Invariant Learning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 264-271, 2003.
[5] D.P. Huttenlocher, G.A. Klanderman, and W.J. Rucklidge, "Comparing Images Using Hausdorff Distance," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 9, pp. 850-863, Sept. 1993.
[6] D. Vukadinovic and M. Pantic, "Fully Automatic Facial Feature Point Detection Using Gabor Feature Based Boosted Classifiers," Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, 2005.
[7] Y. Gizatdinova and V. Surakka, "Feature-Based Detection of Facial Landmarks from Neutral and Expressive Facial Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 135-139, Jan. 2006.
[8] S. Asteriadis, N. Nikolaidis, and I. Pitas, "Facial Feature Detection Using Distance Vector Fields," Pattern Recognition, vol. 42, no. 7, pp. 1388-1398, 2009.
[9] T. Celik, H. Ozkaramanlia, and H. Demirel, "Facial Feature Extraction Using Complex Dual-Tree Wavelet Transform," Computer Vision and Image Understanding, vol. 111, no. 2, pp. 229-246, 2008.
[10] T. Kozakaya, T. Shibata, M. Yuasa, and O. Yamaguchi, "Facial Feature Localization Using Weighted Vector Concentration Approach," Image Vision Computing, vol. 28, no. 5, pp. 772-780, 2010.
[11] T. Wang and P. Shi, "Facial Components Detection with Boosting and Geometric Constraints," Proc. 18th Int'l Conf. Pattern Recognition, 2006.
[12] T.F. Cootes, C.J. Taylor, D.H. Cooper, and J. Graham, "Active Shape Models—Their Training and Application," Computer Vision and Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.
[13] T.F. Cootes, G.J. Edwards, and C.J. Taylor, "Active Appearance Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-685, June 2001.
[14] I. Matthews and S. Baker, "Active Appearance Models Revisited," Int'l J. Computer Vision, vol. 60, no. 2, pp. 135-164, 2004.
[15] D. Cristinacce, T. Cootes, and I. Scott, "A Multi-Stage Approach to Facial Feature Detection," Proc. British Machine Vision Conf., pp. 277-286, 2004.
[16] P. Viola and M. Jones, "Robust Real-Time Face Detection," Int'l J. Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[17] A.A. Salah, H. Cinar, L. Akarun, and B. Sankur, "Robust Facial Landmarking for Registration," Annals of Telecomm., vol. 62, nos. 1/2, pp. 1608-1633, 2007.
[18] M. Valstar, B. Martinez, X. Binefa, and M. Pantic, "Facil Point Detection Using Boosted Regression and Graph Models," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[19] L. Gu and T. Kanade, "A Generative Shape Regularization Model for Robust Face Alignment," Proc. 10th European Conf. Computer Vision, 2008.
[20] D. Cristinacce and T.F. Cootes, "Automatic Feature Localization with Constrained Local Models," Pattern Recognition, vol. 41, no. 10, pp. 3054-3067, 2008.
[21] J.M. Saragih, S. Lucey, and J.F. Cohn, "Deformable Model Fitting by Regularized Landmark Mean-Shift," Int'l J. Computer Vision, vol. 91, no. 2, pp. 200-215, 2011.
[22] P.N. Belhumeur, D.W. Jacobs, D.J. Kriegman, and N. Kumar, "Localizing Parts of Faces Using a Consensus of Exemplars," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 545-552, 2011.
[23] N. Gourier, D. Hall, and J. Crowley, "Facial Features Detection Robust to Pose Illumination and Identity," Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, 2004.
[24] P. Sankaran, S. Gundimada, R. Tompkins, and V. Asari, "Pose Angle Determination by Face Eyes and Nose Localization," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, 2005.
[25] T. Cootes, K. Walker, and C. Taylor, "View-Based Active Appearance Models," Image and Vision Computing, vol. 20, pp. 657-664, 2002.
[26] Y. Tong and Q. Ji, "Multiview Facial Feature Tracking with a Multi-Modal Probabilistic Model," Proc. 18th Int'l Conf. Pattern Recognition, pp. 307-310, 2006.
[27] Y. Tong, Y. Wang, Z. Zhu, and Q. Ji, "Robust Facial Feature Tracking under Varying Face Pose And Facial Expression," Pattern Recognition, vol. 40, no. 11, pp. 3195-3208, 2007.
[28] S.M. Hanif, L. Prevost, R. Belaroussi, and M. Milgram, "Real-Time Facial Feature Localization by Combining Space Displacement Neural Networks," Pattern Recognition Letters, vol. 29, no. 8, pp. 1094-1104, 2008.
[29] C. Christoudias and T. Darrell, "On Modelling Nonlinear Shape-and-Texture Appearance Manifolds," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1067-1074, 2005.
[30] S. Romdhani, S. Gong, and A. Psarrou, "A Multi-View Nonlinear Active Shape Model Using Kernel PCA," Proc. British Machine Vision Conf., pp. 483-499, 1999.
[31] T. Vetter and V. Blanz, "Estimating Coloured 3D Face Models from Single Images: An Example Based Approach," Proc. European Conf. Computer Vision, pp. 499-513, 1998.
[32] P. Paysan, R. Knothe, B. Amberg, S. Romdhani, and T. Vetter, "A 3D Face Model for Pose and Illumination Invariant Face Recognition," Proc. Sixth IEEE Int'l Conf. Advanced Video and Signal Based Surveillance, pp. 296-301, 2009.
[33] J. Xiao, S. Baker, I. Matthews, and T. Kanade, "Real-Time Combined 2D+3D Active Appearance Models," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 535-542, 2004.
[34] A. Caunce, D. Cristinacce, C.J. Taylor, and T.F. Cootes, "Locating Facial Features and Pose Estimation Using a 3D Shape Model," Proc. Int'l Symp. Visual Computing, 2009.
[35] L. Gu and T. Kanade, "3D Alignment of Face in a Single Image," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[36] S. Romdhani and T. Vetter, "3D Probabilistic Feature Point Model for Object Detection and Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[37] H. Bay, A. Ess, T. Tuytelaars, and L. van Gool, "SURF: Speeded Up Robust Features," Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346-359, 2008.
[38] M. Agrawal, K. Konolige, and M.R. Blas, "CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching," Proc. European Conf. Computer Vision, 2008.
[39] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[40] L. Fei-Fei and P. Perona, "A Bayesian Hierarchical Model for Learning Natural Scene Categories," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[41] S. Lazebnik, C. Schmid, and J. Ponce, "Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2169-2178, 2006.
[42] C. Bishop, Neural Networks for Pattern Recognition. Oxford Univ. Press, 1995.
[43] Masschusetts Inst. of Technology and Center for Biological and Computational Learning, "MIT-CBCL Face Recognition Database. Copyright 2003-2005 Massachusetts Inst. of Technology," http://cbcl.mit.edu/software-datasets/heisele facerecognition-database. html, 2012.
[44] B. Weyrauch, J. Huang, B. Heisele, and V. Blanz, "Component-Based Face Recognition with 3D Morphable Models," Proc. First IEEE Workshop Face Processing in Video, 2004.
[45] P. Phillips, H. Wechsler, J. Huang, and P. Rauss, "The FERET Database and Evaluation Procedure for Face Recognition Algorithms," Image and Vision Computing, vol. 16, no. 5, pp. 295-306, 1998.
[46] P.J. Phillips, H. Moon, S.A. Rizvi, and P. Rauss, "The FERET Evaluation Methodology for Face Recognition Algorithms," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090-1104, Oct. 2000.
[47] PICS, "Psychological Image Collection at Stirling," http:/pics.psych.stir.ac.uk/, 2012.
[48] "Georgia Tech Face Database," http://www.anefian.com research/, 2012.
[49] Equinox Corp., "DARPA's HumanID Program," http://www. equinoxsensors.com/productsHID.html , 2012.
[50] A. Selinger and D. Socolinsky, "Appearance-Based Facial Recognition Using Visible and Thermal Imagery: A Comparative Study," Technical Report 0201, Equinox Corp., 2002.
[51] "Yale Face Database B," http://cvc.yale.edu/projects/ yalefaces B yalefacesB.html, 2012.
[52] A. Georghiades, P. Belhumeur, and D. Kriegman, "From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001.
[53] "CUbiC FacePix(30) Database," http:/www.facepix.org/, 2012.
[54] J. Black, M. Gargesha, K. Kahol, and S. Panchanathan, "A Framework for Performance Evaluation of Face Recognition Algorithms," Proc. Conf. Internet Multimedia Systems II, 2002.
[55] G. Little, S. Krishna, J. Black, and S. Panchanathan, "A Methodology for Evaluating Robustness of Face Fecognition Algorithms with Respect to Changes in Pose and Illumination Angle," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, 2005.
[56] C. Conde, A. Serrano, and E. Cabello, "Multimodal 2D, 2.5D and 3D Face Verification," Proc. IEEE Int'l Conf. Image Processing, pp. 2061-2064, 2006.
[57] A. Pentland, B. Moghaddam, and T. Starner, "View-Based and Modular Eigenspaces for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994.
[58] C. Micheloni, E. Sangineto, L. Cinque, and G.L. Foresti, "Improved Statistical Techniques for Multi-Part Face Detection and Recognition," Proc. 16th Scandinavian Conf. Image Analysis, 2009.
[59] D.A. Forsyth and J. Ponce, Computer Vision: A Modern Approach. Prentice Hall, 2003.
[60] T. Jebara and A. Pentland, "Parametrized Structure from Motion for 3D Adaptive Feedback Tracking of Faces," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.
[61] L. Chen, L. Zhang, H. Zhang, and M. Abdel-Mottaleb, "3D Shape Constraint for Facial Feature Localization Using Probabilistic-Like Output," Proc. Int'l Conf. Automatic Face and Gesture Recognition, 2004.
[62] D.H. Ballard and C.M. Brown, Computer Vision. Prentice-Hall, 1982.
[63] P.F. Felzenszwalb and D.P. Huttenlocher, "Distance Transforms of Sampled Functions," technical report, Cornell Computing and Information Science, 2004.
[64] E. Tola, V. Lepetit, and P. Fua, "DAISY: An Efficient Dense Descriptor Applied to Wide Baseline Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 815-830, May 2010.
[65] M. Calonder, V. Lepetit, C. Strecha, and P. Fua, "BRIEF: Binary Robust Independent Elementary Features," Proc. European Conf. Computer Vision, 2010.
[66] C. Evans, "Notes on the OpenSURF Library," Technical Report CSTR-09-001, Univ. of Bristol, Jan. 2009.
[67] R. Hess, "OpenCV Implementation of the SIFT Features," technical report, http://eecs.oregonstate.edu/hess/worksift.html , 2007.
[68] K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre, "XM2VTSdb: The Extended M2Vts Database," Proc. Int'l Conf. Audio and Video-Base Biometric Personal Verification, 1999.
[69] O. Jesorsky, K.J. Kirchberg, and R.W. Frischholz, "Robust Face Detection Using the Hausdorff Distance," Proc. Int'l Conf. Audio and Video-Based Biometric Authentication, 2001.
[70] N. Gourier, D. Hall, and J. Crowley, "Estimating Face Orientation form Robust Detection of Salient Facial structures," Proc. ICPR Workshop Visual Observation of Deictic Gestures, 2004.
43 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool