The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - October (2011 vol.33)
pp: 1938-1951
T. Theoharis , Dept. of Inf. & Telecommun., Univ. of Athens, Athens, Greece
P. Perakis , Dept. of Inf. & Telecommun., Univ. of Athens, Athens, Greece
G. Passalis , Dept. of Inf. & Telecommun., Univ. of Athens, Athens, Greece
I. A. Kakadiaris , Dept. of Comput. Sci., Univ. of Houston, Houston, TX, USA
ABSTRACT
The uncontrolled conditions of real-world biometric applications pose a great challenge to any face recognition approach. The unconstrained acquisition of data from uncooperative subjects may result in facial scans with significant pose variations along the yaw axis. Such pose variations can cause extensive occlusions, resulting in missing data. In this paper, a novel 3D face recognition method is proposed that uses facial symmetry to handle pose variations. It employs an automatic landmark detector that estimates pose and detects occluded areas for each facial scan. Subsequently, an Annotated Face Model is registered and fitted to the scan. During fitting, facial symmetry is used to overcome the challenges of missing data. The result is a pose invariant geometry image. Unlike existing methods that require frontal scans, the proposed method performs comparisons among interpose scans using a wavelet-based biometric signature. It is suitable for real-world applications as it only requires half of the face to be visible to the sensor. The proposed method was evaluated using databases from the University of Notre Dame and the University of Houston that, to the best of our knowledge, include the most challenging pose variations publicly available. The average rank-one recognition rate of the proposed method in these databases was 83.7 percent.
INDEX TERMS
wavelet transforms, face recognition, pose estimation, stereo image processing, wavelet-based biometric signature, facial symmetry, pose variation, 3D face recognition, biometric application, unconstrained data acquisition, uncooperative subjects, facial scan, yaw axis, landmark detection, pose estimation, occluded area detection, annotated face model, pose invariant geometry image, interpose scan, Shape, Face, Three dimensional displays, Nose, Face recognition, Indexes, physically-based modeling., Biometrics, face and gesture recognition
CITATION
T. Theoharis, P. Perakis, G. Passalis, I. A. Kakadiaris, "Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 10, pp. 1938-1951, October 2011, doi:10.1109/TPAMI.2011.49
REFERENCES
[1] G. Passalis, I. Kakadiaris, and T. Theoharis, "Intraclass Retrieval of Non-Rigid 3D Objects: Application to Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 218-229, Feb. 2007.
[2] I. Kakadiaris, G. Passalis, G. Toderici, M. Murtuza, Y. Lu, N. Karampatziakis, and T. Theoharis, "Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 4, pp. 640-649, Apr. 2007.
[3] P. Perakis, G. Passalis, T. Theoharis, G. Toderici, and I. Kakadiaris, "Partial Matching of Interpose 3D Facial Data for Face Recognition," Proc. Third IEEE Int'l Conf. Biometrics: Theory, Applications, and Systems, pp. 439-446, Sept. 2009.
[4] P. Perakis, T. Theoharis, G. Passalis, and I. Kakadiaris, "Automatic 3D Facial Region Retrieval from Multi-Pose Facial Datasets," Proc. Eurographics Workshop 3D Object Retrieval, pp. 37-44, Mar./Apr. 2009.
[5] P. Perakis, T. Theoharis, G. Passalis, and I. Kakadiaris, "3D Facial Landmark Detection & Face Registration: A 3D Facial Landmark Model & 3D Local Shape Descriptors Approach," technical report, Computer Graphics Laboratory, Univ. of Athens, http://graphics. di.uoa.gr/Research/TechReports 3D_Lmk_v1.pdf, Jan. 2011.
[6] K. Bowyer, K. Chang, and P. Flynn, "A Survey of Approaches and Challenges in 3D and Multi-Modal 3D+2D Face Recognition," Computer Vision and Image Understanding, vol. 101, no. 1, pp. 1-15, Jan. 2006.
[7] K. Chang, K. Bowyer, and P.J. Flynn, "An Evaluation of Multi-Modal 2D+3D Face Biometrics," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 619-624, Apr. 2005.
[8] X. Lu and A. Jain, "Multimodal Facial Feature Extraction for Automatic 3D Face Recognition," Technical Report MSU-CSE-05-22, Michigan State Univ., Oct. 2005.
[9] X. Lu and A. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition," Proc. Seventh Int'l Conf. Automatic Face and Gesture Recognition, pp. 585-590, Apr. 2006.
[10] X. Lu, A. Jain, and D. Colbry, "Matching 2.5D Face Scans to 3D Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 31-43, Jan. 2006.
[11] H. Dibeklioglu, "Part-Based 3D Face Recognition under Pose and Expression Variations," master's thesis, Bogazici Univ., 2008.
[12] H. Dibeklioglu, A. Salah, and L. Akarun, "3D Facial Landmarking under Expression, Pose and Occlusion Variations," Proc. Second Int'l Conf. Biometrics Theory, Applications and Systems, pp. 1-6, Sept./Oct. 2008.
[13] V. Blanz and T. Vetter, "Face Recognition Based on Fitting a 3D Morphable Model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[14] V. Blanz, K. Scherbaum, and H.-P. Seidel, "Fitting a Morphable Model to 3D Scans of Faces," Proc. 11th IEEE Int'l Conf. Computer Vision, pp. 1-8, Oct. 2007.
[15] A. Bronstein, M. Bronstein, and R. Kimmel, "Robust Expression-Invariant Face Recognition from Partially Missing Data," Proc. European Conf. Computer Vision, pp. 396-408, 2006.
[16] P. Nair and A. Cavallaro, "Matching 3D Faces with Partial Data," Proc. British Machine Vision Conf., Sept. 2008.
[17] P. Nair and A. Cavallaro, "3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model," IEEE Trans. Multimedia, vol. 11, no. 4, pp. 611-623, June 2009.
[18] T. Lin, W. Shih, W. Chen, and W. Ho, "3D Face Authentication by Mutual Coupled 3D and 2D Feature Extraction," Proc. 44th ACM Southeast Regional Conf., pp. 423-427, Mar. 2006.
[19] A. Mian, M. Bennamoun, and R. Owen, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1927-1943, Nov. 2007.
[20] M. Segundo, C. Queirolo, O. Bellon, and L. Silva, "Automatic 3D Facial Segmentation and Landmark Detection," Proc. 14th Int'l Conf. Image Analysis and Processing, pp. 431-436, Sept. 2007.
[21] X. Wei, P. Longo, and L. Yin, "Automatic Facial Pose Determination of 3D Range Data for Face Model and Expression Identification," Advances in Biometrics, pp. 144-153, Springer, 2007.
[22] T. Faltemier, K. Bowyer, and P. Flynn, "Rotated Profile Signatures for Robust 3D Feature Detection," Proc. Eighth IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 1-7, Sept. 2008.
[23] C. Xu, T. Tan, Y. Wang, and L. Quan, "Combining Local Features for Robust Nose Location in 3D Facial Data," Pattern Recognition Letters, vol. 27, no. 13, pp. 62-73, 2006.
[24] Y. Liu and J. Palmer, "A Quantified Study of Facial Asymmetry in 3D Faces," Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, pp. 222-229, Oct. 2003.
[25] L. Kompanets, "Biometrics of Asymmetrical Face," Proc. First Int'l Conf. Biometric Authentication, pp. 67-73, July 2004.
[26] Y. Liu, R.L. Weaver, K. Schmidt, N. Serban, and J. Cohn, "Facial Asymmetry: A New Biometric," Technical Report CMU-RI-TR-01-23, Robotics Inst., Carnegie Mellon Univ. Aug. 2001.
[27] S. Mitra, M. Savvides, and B.V. Kumar, "Face Identification Using Novel Frequency-Domain Representation of Facial Asymmetry," IEEE Trans. Information Forensics and Security, vol. 1, no. 3, pp. 350-359, Sept. 2006.
[28] S. Gutta, V. Philomin, and M. Trajkovic, "An Investigation into the Use of Partial-Faces for Face Recognition," Proc. Fifth IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 28-33, May 2002.
[29] I. Dryden and K. Mardia, Statistical Shape Analysis. Wiley, 1998.
[30] M.B. Stegmann and D.D. Gomez, "A Brief Introduction to Statistical Shape Analysis," technical report, Technical Univ. of Denmark, Mar. 2002.
[31] T. Cootes and C. Taylor, "Statistical Models of Appearance for Computer Vision," technical report, Univ. of Manchester, Oct. 2001.
[32] T. Cootes, C. Taylor, D. Cooper, and J. Graham, "Active Shape Models—Their Training and Application," Computer Vision and Image Understanding, vol. 61, no. 1, pp. 38-59, Jan. 1995.
[33] T. Cootes, C. Taylor, H. Kang, and V. Petrovic, "Modeling Facial Shape and Appearance," Handbook of Face Recognition, pp. 39-63, Springer, 2005.
[34] D. Colbry, "Human Face Verification by Robust 3D Surface Alignment," PhD dissertation, Michigan State Univ., 2006.
[35] D. Colbry, G. Stockman, and A. Jain, "Detection of Anchor Points for 3D Face Verification," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition Workshops, p. 118, June 2005.
[36] A. Johnson, "Spin-Images: A Representation for 3-D Surface Matching," PhD dissertation, Robotics Inst., Carnegie Mellon Univ., Aug. 1997.
[37] L. Farkas, Anthropometry of the Head and Face, L.G. Farkas, ed., second ed. Raven Press, 1994.
[38] X. Gu, S. Gortler, and H. Hoppe, "Geometry Images," Proc. ACM SIGGRAPH, pp. 355-361, July 2002.
[39] E. Praun and H. Hoppe, "Spherical Parametrization and Remeshing," Proc. ACM SIGGRAPH, pp. 340-349, July 2003.
[40] G. Papaioannou, E. Karabassi, and T. Theoharis, "Reconstruction of Three-Dimensional Objects through Matching of Their Parts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 114-124, Jan. 2002.
[41] S. Kirkpatrick, C. Gelatt, and M. Vecchi, "Optimization by Simulated Annealing," Science, vol. 22, no. 4598, pp. 671-680, 1983.
[42] P. Siarry, G. Berthiau, F. Durbin, and J. Haussy, "Enhanced Simulated Annealing for Globally Minimizing Functions of Many-Continuous Variables," ACM Trans. Math. Software, vol. 23, no. 2, pp. 209-228, 1997.
[43] C. Mandal, "A Dynamic Framework for Subdivision Surfaces," PhD dissertation, Univ. of Florida, 1998.
[44] C. Loop, "Smooth Subdivision Surfaces Based on Triangles," master's thesis, Dept. of Math., Univ. of Utah, 1987.
[45] A. Klinger, "Pattern and Search Statistics," Proc. Optimizing Methods in Statistics, pp. 303-337, 1971.
[46] A. Finkel and J. Bentley, "Quad Trees: A Data Structure for Retrieval of Composite Keys," Acta Informatica, vol. 4, no. 1, pp. 1-9, 1974.
[47] E. Stollnitz, T. DeRose, and D. Salesin, Wavelets for Computer Graphics: Theory and Applications. Morgan Kaufmann Publishers, Inc., 1996.
[48] P. Phillips, P. Flynn, T. Scruggs, K. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, "Overview of the Face Recognition Grand Challenge," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 947-954, 2005.
[49] P. Phillips, W. Scruggs, A. OToole, P. Flynn, K. Bowyer, C. Schott, and M. Sharpe, "FRVT 2006 and ICE 2006 Large-Scale Experimental Results," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 831-846, May 2010.
[50] U. of Notre Dame, "University of Notre Dame Biometrics Database," http://www.nd.edu/~cvrlUNDBiometricsDatabase. html , 2008.
[51] "UH Biometrics Databases," http://cbl.uh.edu/URxDdatasets/, 2011.
[52] Y. Gao, "Efficiently Comparing Face Images Using a Modified Hausdorff Distance," Proc. IEEE Conf. Vision, Image, and Signal Processing, pp. 346-350, Dec. 2003.
17 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool