The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2013 vol.35)
pp: 1552-1564
P. Perakis , Dept. of Inf. & Telecommun., Univ. of Athens, Ilisia, Greece
G. Passalis , Dept. of Inf. & Telecommun., Univ. of Athens, Ilisia, Greece
T. Theoharis , Dept. of Inf. & Telecommun., Univ. of Athens, Ilisia, Greece
I. A. Kakadiaris , Dept. of Comput. Sci., Univ. of Houston, Houston, TX, USA
ABSTRACT
A 3D landmark detection method for 3D facial scans is presented and thoroughly evaluated. The main contribution of the presented method is the automatic and pose-invariant detection of landmarks on 3D facial scans under large yaw variations (that often result in missing facial data), and its robustness against large facial expressions. Three-dimensional information is exploited by using 3D local shape descriptors to extract candidate landmark points. The shape descriptors include the shape index, a continuous map of principal curvature values of a 3D object's surface, and spin images, local descriptors of the object's 3D point distribution. The candidate landmarks are identified and labeled by matching them with a Facial Landmark Model (FLM) of facial anatomical landmarks. The presented method is extensively evaluated against a variety of 3D facial databases and achieves state-of-the-art accuracy (4.5-6.3 mm mean landmark localization error), considerably outperforming previous methods, even when tested with the most challenging data.
INDEX TERMS
Shape, Nose, Face, Indexes, Feature extraction, Eigenvalues and eigenfunctions,spin images, Face models, landmark detection, shape index
CITATION
P. Perakis, G. Passalis, T. Theoharis, I. A. Kakadiaris, "3D Facial Landmark Detection under Large Yaw and Expression Variations", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 7, pp. 1552-1564, July 2013, doi:10.1109/TPAMI.2012.247
REFERENCES
[1] I. Kakadiaris, G. Passalis, G. Toderici, M. Murtuza, Y. Lu, N. Karampatziakis, and T. Theoharis, "Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 4, pp. 640-649, Apr. 2007.
[2] G. Passalis, P. Perakis, T. Theoharis, and I. Kakadiaris, "Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 1938-1951, Oct. 2011.
[3] P. Perakis, T. Theoharis, G. Passalis, and I. Kakadiaris, "Automatic 3D Facial Region Retrieval from Multi-Pose Facial Data Sets," Proc. Eurographics Workshop 3D Object Retrieval, pp. 37-44, Mar./Apr. 2009.
[4] P. Perakis, G. Passalis, T. Theoharis, G. Toderici, and I. Kakadiaris, "Partial Matching of Interpose 3D Facial Data for Face Recognition," Proc. Third IEEE Int'l Conf. Biometrics: Theory, Applications and Systems, pp. 439-446, Sept. 2009.
[5] P. Perakis, G. Passalis, T. Theoharis, and I. Kakadiaris, "3D Facial Landmark Detection & Face Registration: A 3D Facial Landmark Model & 3D Local Shape Descriptors Approach," Technical Report TP-2010-01, Computer Graphics Laboratory, Univ. of Athens, Jan. 2010.
[6] P. Phillips, P. Flynn, T. Scruggs, K. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, "Overview of the Face Recognition Grand Challenge," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 947-954, 2005.
[7] P. Phillips, T. Scruggs, A. O'Toole, P. Flynn, K. Bowyer, C. Schott, and M. Sharpe, "FRVT 2006 and ICE 2006 Large-Scale Experimental Results," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 831-846, May 2010.
[8] UND, "University of Notre Dame Biometrics Data Sets," http://www.nd.edu/~cvrl/CVRLData_Sets.html , 2012.
[9] X. Lu and A. Jain, "Multimodal Facial Feature Extraction for Automatic 3D Face Recognition," Technical Report MSU-CSE-05-22, Michigan State Univ., Oct. 2005.
[10] D. Colbry, G. Stockman, and A. Jain, "Detection of Anchor Points for 3D Face Verification," Proc. IEEE Conf. Computer Vision and Pattern Recognition, p. 118, June 2005.
[11] X. Lu and A. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition," Proc. Seventh Int'l Conf. Automatic Face and Gesture Recognition, pp. 585-590, 2006.
[12] X. Lu, A. Jain, and D. Colbry, "Matching 2.5D Face Scans to 3D Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 31-43, Jan. 2006.
[13] D. Colbry, "Human Face Verification by Robust 3D Surface Alignment," PhD dissertation, Michigan State Univ., 2006.
[14] C. Dorai and A.K. Jain, "COSMOS—A Representation Scheme for 3D Free-Form Objects," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 10, pp. 1115-1130, Oct. 1997.
[15] C. Harris and M. Stephens, "A Combined Corner and Edge Detector," Proc. Fourth Alvey Vision Conf., pp. 147-151, 1988,
[16] C. Conde, R. Cipolla, L.J. Rodrígez-Aragón, A. Serrano, and E. Cabello, "3D Facial Feature Location with Spin Images," Proc. IAPR Conf. Machine Vision Applications, pp. 418-421, May 2005.
[17] C. Xu, T. Tan, Y. Wang, and L. Quan, "Combining Local Features for Robust Nose Location in 3D Facial Data," Pattern Recognition Letters, vol. 27, no. 13, pp. 62-73, 2006.
[18] T. Lin, W. Shih, W. Chen, and W. Ho, "3D Face Authentication by Mutual Coupled 3D and 2D Feature Extraction," Proc. 44th ACM Southeast Regional Conf., pp. 423-427, Mar. 2006.
[19] M. Segundo, C. Queirolo, O. Bellon, and L. Silva, "Automatic 3D Facial Segmentation and Landmark Detection," Proc. 14th Int'l Conf. Image Analysis and Processing, pp. 431-436, Sept. 2007.
[20] X. Wei, P. Longo, and L. Yin, "Automatic Facial Pose Determination of 3D Range Data for Face Model and Expression Identification," Proc. Int'l Conf. Advances in Biometrics, pp. 144-153, 2007.
[21] A. Mian, M. Bennamoun, and R. Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1927-1943, Nov. 2007.
[22] T. Faltemier, K. Bowyer, and P. Flynn, "A Region Ensemble for 3-D Face Recognition," IEEE Trans. Information Forensics and Security, vol. 3, no. 1, pp. 62-73, Mar. 2008.
[23] T. Faltemier, K. Bowyer, and P. Flynn, "Rotated Profile Signatures for Robust 3D Feature Detection," Proc. IEEE Eighth Int'l Conf. Automatic Face and Gesture Recognition, pp. 1-7, Sept. 2008.
[24] H. Dibeklioğlu, "Part-Based 3D Face Recognition under Pose and Expression Variations," master's thesis, Boğaziçi Univ., 2008.
[25] H. Dibeklioğlu, A. Salah, and L. Akarun, "3D Facial Landmarking under Expression, Pose, and Occlusion Variations," Proc. Second IEEE Int'l Conf. Biometrics: Theory, Applications and Systems, pp. 1-6, Sept./Oct. 2008.
[26] T. Yu and Y. Moon, "A Novel Genetic Algorithm for 3D Facial Landmark Localization," Proc. Second IEEE Int'l Conf. Biometrics: Theory, Applications, and Systems, Sept./Oct. 2008.
[27] M. Romero-Huertas and N. Pears, "3D Facial Landmark Localization by Matching Simple Descriptors," Proc. Second IEEE Int'l Conf. Biometrics: Theory, Applications, and Systems, Sept./Oct. 2008.
[28] P. Nair and A. Cavallaro, "3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model," IEEE Trans. Multimedia, vol. 11, no. 4, pp. 611-623, June 2009.
[29] L. Yin, X. Wei, Y. Sun, J. Wang, and M. Rosato, "A 3D Facial Expression Database for Facial Behavior Research," Proc. Seventh Int'l Conf. Automatic Face and Gesture Recognition, pp. 211-216, Apr. 2006.
[30] I. Dryden and K. Mardia, Statistical Shape Analysis. Wiley, 1998.
[31] M. Stegman and D. Gomez, "A Brief Introduction to Statistical Shape Analysis," technical report, Technical Univ. of Denmark, Mar. 2002.
[32] T. Cootes and C. Taylor, "Statistical Models of Appearance for Computer Vision," technical report, Univ. of Manchester, Oct. 2001.
[33] T. Cootes, C. Taylor, H. Kang, and V. Petrovic, "Modeling Facial Shape and Appearance," Handbook of Face Recognition, pp. 39-63, Springer, 2005.
[34] S. Theodoridis and K. Koutroumbas, Pattern Recognition, third ed. Academic Press, 2006.
[35] J. Koenderink and A. van Doorn, "Surface Shape and Curvature Scales," Image and Vision Computing, vol. 10, pp. 557-565, Oct. 1992.
[36] A.E. Johnson, "Spin Images: A Representation for 3-D Surface Matching," PhD dissertation, Robotics Inst., Carnegie Mellon Univ., Pittsburgh, Penn., Aug. 1997.
[37] UH-CBL, "Facial Landmarks Annotation Files," http://www.cbl.uh.edu/URxD/annotationsfacial-landmarks.zip , ver. 3, 2012.
30 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool