This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition
November 2007 (vol. 29 no. 11)
pp. 1927-1943
ASCII Text
x 
 
Ajmal Mian, Mohammed Bennamoun, Robyn Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1927-1943, November, 2007.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2007.1105,
author = {Ajmal Mian and Mohammed Bennamoun and Robyn Owens},
title = {An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {29},
number = {11},
issn = {0162-8828},
year = {2007},
pages = {1927-1943},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2007.1105},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition
IS - 11
SN - 0162-8828
SP1927
EP1943
EPD - 1927-1943
A1 - Ajmal Mian,
A1 - Mohammed Bennamoun,
A1 - Robyn Owens,
PY - 2007
KW - Biometrics
KW - face recognition
KW - rejection classifier
KW - 3D shape representation
VL - 29
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
We present a fully automatic face recognition algorithm and demonstrate its performance on the FRGC v2.0 data. Our algorithm is multimodal (2D and 3D) and performs hybrid (feature-based and holistic) matching in order to achieve efficiency and robustness to facial expressions. The pose of a 3D face along with its texture is automatically corrected using a novel approach based on a single automatically detected point and the Hotelling transform. A novel 3D Spherical Face Representation (SFR) is used in conjunction with the SIFT descriptor to form a rejection classifier which quickly eliminates a large number of candidate faces at an early stage for efficient recognition in case of large galleries. The remaining faces are then verified using a novel region-based matching approach which is robust to facial expressions. This approach automatically segments the eyes-forehead and the nose regions, which are relatively less sensitive to expressions, and matches them separately using a modified ICP algorithm. The results of all the matching engines are fused at the metric level to achieve higher accuracy. We use the FRGC benchmark to compare our results to other algorithms which used the same database. Our multimodal hybrid algorithm performed better than others by achieving 99.74% and 98.31% verification rates at 0.001 FAR and identification rates of 99.02% and 95.37% for probes with neutral and non-neutral expression respectively.

[1] K. Arun, T. Huang, and S. Blostein, “Least-Squares Fitting of Two 3-D Point Sets,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 5, pp. 698-700, 1987.
[2] S. Baker and S.K. Nayar, “Pattern Rejection,” Proc. IEEE Computer Vision and Pattern Recognition, pp. 544-549, 1996.
[3] M.S. Bartlett, H.M. Lades, and T. Sejnowski, “Independent Component Representation for Face Recognition,” Proc. SPIE Symp. Electronic Imaging, pp. 528-539, 1998.
[4] P. Belhumeur, J. Hespanha, and D. Kriegman, “Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, pp.711-720, 1997.
[5] P.J. Besl and N.D. McKay, “Reconstruction of Real-World Objects via Simultaneous Registration and Robust Combination of Multiple Range Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[6] Biometric Consortium, http:/www.biometrics.org, 2004.
[7] K.W. Bowyer, K. Chang, and P. Flynn, “A Survey of Approaches and Challenges in 3D and Multi-Modal 3D + 2D Face Recognition,” Computer Vision and Image Understanding, vol. 101, pp. 1-15, 2006.
[8] A.M. Bronstein, M.M. Bronstein, and R. Kimmel, “Three-Dimensional Face Recognition,” Int'l J. Computer Vision, vol. 64, no. 1, pp.5-30, 2005.
[9] K.I. Chang, K.W. Bowyer, and P.J. Flynn, “Face Recognition Using 2D and 3D Facial Data,” Proc. Workshop Multimodal User Authentication, pp. 25-32, 2003.
[10] K.I. Chang, K.W. Bowyer, and P.J. Flynn, “Multi-Modal 2D and 3D Biometrics for Face Recognition,” Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, pp. 187-194, 2003.
[11] K.I. Chang, K.W. Bowyer, and P.J. Flynn, “ARMS: Adaptive Rigid Multi-Region Selection for Handling Expression Variation in 3D Face Recognition,” Proc. IEEE Workshop Face Recognition Grand Challenge (FRGC) Experiments, 2005.
[12] C.S. Chua and R. Jarvis, “Point Signatures: A New Representation for 3D Object Recognition,” Int'l J. Computer Vision, vol. 25, no. 1, pp. 63-85, 1997.
[13] C. Chua, F. Han, and Y. Ho, “3D Human Face Recognition Using Point Signatures,” Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, pp. 233-238, 2000.
[14] I.J. Cox, J. Ghosn, and P.N. Yianilos, “Feature-Based Face Recognition Using Mixture Distance,” Proc. IEEE Computer Vision and Pattern Recognition, pp. 209-216, 1996.
[15] Geometrix, http:/www.geometrix.com/, 2006.
[16] R.C. Gonzalez and R.E. Woods, Digital Image Processing. Addison-Wesley, 1992.
[17] J. Huang, B. Heisele, and V. Blanz, “Component-Based Face Recognition with 3D Morphable Models,” Proc. Int'l Conf. Audio- and Video-Based Person Authentication, 2003.
[18] M. Huskën, M. Brauckmann, S. Gehlen, and C. Malsburg, “Strategies and Benefits of Fusion of 2D and 3D Face Recognition,” Proc. IEEE Workshop Face Recognition Grand Challenge Experiments, 2005.
[19] A.K. Jain, L. Hong, and S. Pankanti, “Biometric Identification,” Comm. ACM, vol. 43, no. 2, pp. 91-98, 2000.
[20] A.K. Jain, A. Ross, and S. Prabhakar, “An Introduction to Biometric Recognition,” IEEE Trans. Circuits and Systems for Video Technology, vol. 14, no. 1, pp. 4-20, 2004.
[21] A.E. Johnson and M. Hebert, “Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 674-686, May 1999.
[22] P. Kovesi, “MATLAB and Octave Functions for Computer Vision and Image Processing,” http://people.csse.uwa.edu.au/pk/Research/ MatlabFnsindex.html, 2006.
[23] A. Lanitis, C. Taylor, and T. Cootes, “Automatic Face Identification System Using Flexible Appearance Models,” Image and Vision Computing, vol. 13, pp. 393-401, 1995.
[24] D. Lin and X. Tang, “Recognize High Resolution Faces: From Macrocosm to Microcosm,” Proc. IEEE Computer Vision and Pattern Recognition, pp. 1355-1362, 2006.
[25] C. Liu and H. Wechsler, “Evolutionary Pursuit and Its Application to Face Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, pp. 570-582, 2000.
[26] D. Lowe, “Distinctive Image Features from Scale-Invariant Key Points,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[27] D. Lowe, “Demo Software: SIFT Keypoint Detector,” http://www.cs.ubc.edu.ca~lowe/, 2006.
[28] X. Lu, A.K. Jain, and D. Colbry, “Matching 2.5D Scans to 3D Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 31-43, Jan. 2006.
[29] T. Maurer, D. Guigonis, I. Maslov, B. Pesenti, A. Tsaregorodtsev, D. West, and G. Medioni, “Performance of Geometrix ActiveID 3D Face Recognition Engine on the FRGC Data,” Proc. IEEE Workshop Face Recognition Grand Challenge Experiments, 2005.
[30] G. Medioni and R. Waupotitsch, “Face Recognition and Modeling in 3D,” Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, pp. 232-233, 2003.
[31] A.S. Mian, M. Bennamoun, and R.A. Owens, “A Novel Representation and Feature Matching Algorithm for Automatic Pairwise Registration of Range Images,” Int'l J. Computer Vision, vol. 66, pp. 19-40, 2006.
[32] A.S. Mian, M. Bennamoun, and R.A. Owens, “Region-Based Matching for Robust 3D Face Recognition,” Proc. British Machine Vision Conf., vol. 1, pp. 199-208, 2005.
[33] A.S. Mian, M. Bennamoun, and R.A. Owens, “2D and 3D Multimodal Hybrid Face Recognition,” Proc. European Conf. Computer Vision, part III, pp. 344-355, 2006.
[34] A.S. Mian, M. Bennamoun, and R.A. Owens, “Automatic 3D Face Detection, Normalization and Recognition,” Proc. Third Int'l Symp. 3D Data Processing, Visualization and Transmission, 2006.
[35] A.S. Mian, M. Bennamoun, and R.A. Owens, “Three-Dimensional Model-Based Object Recognition and Segmentation in Cluttered Scenes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1584-1601, Oct. 2006.
[36] Minolta 3D Digitizers, Non-Contact 3D Laser Scanner, http:/www.minolta3d.com, 2006.
[37] B. Moghaddam and A. Pentland, “Probabilistic Visual Learning for Object Representation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, pp. 696-710, 1997.
[38] G. Passalis, I. Kakadiaris, T. Theoharis, G. Tederici, and N. Murtaza, “Evaluation of 3D Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach,” Proc. IEEE Workshop Face Recognition Grand Challenge Experiments, 2005.
[39] A. Pentland, B. Moghaddam, and T. Starner, “View-Based and Modular Eigenspaces for Face Recognition,” Proc. IEEE Computer Vision and Pattern Recognition, 1994.
[40] P.J. Phillips, “Support Vector Machines Applied to Face Recognition,” Proc. 1998 Conf. Advances in Neural Information Processing Systems, vol. 11, pp. 803-809, 1998.
[41] P.J. Phillips, P.J. Flynn, T. Scruggs, K. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, “Overview of the Face Recognition Grand Challenge,” Proc. IEEE Computer Vision and Pattern Recognition, 2005.
[42] P.J. Phillips, P.J. Flynn, T. Scruggs, K.W. Bowyer, and W. Worek, “Preliminary Face Recognition Grand Challenge Results,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, 2006.
[43] F. Samaria and S. Young, “HMM Based Architecture for Face Identification,” Int'l J. Image and Vision Computing, vol. 12, pp. 537-583, 1994.
[44] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J.Cognitive Neuroscience, vol. 3, 1991.
[45] P. Viola and M.J. Jones, “Robust Real-Time Face Detection,” Int'l J.Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[46] Y. Wang, C. Chua, and Y. Ho, “Face Recognition From 2D and 3D Images,” Proc. Int'l Conf. Audio and Video-Based Person Authentication, pp. 26-31, 2001.
[47] L. Wiskott, J. Fellous, and C. Malsburg, “Face Recognition by Elastic Bunch Graph Matching,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, pp. 775-779, 1997.
[48] C. Xu, Y. Wang, T. Tan, and L. Quan, “Automatic 3D Face Recognition Combining Global Geometric Features with Local Shape Variation Information,” Proc. IEEE Int'l Conf. Pattern Recognition, pp. 308-313, 2004.
[49] P. Yan and K. Bowyer, “Empirical Evaluation of Advanced Ear Biometrics,” Proc. IEEE Computer Vision and Pattern Recognition Workshops, pp. 308-313, 2004.
[50] W. Zhao, R. Chellappa, P.J. Phillips, and A. Rosenfeld, “Face Recognition: A Literature Survey,” ACM Computing Surveys, pp.399-458, 2003.

Index Terms:
Biometrics, face recognition, rejection classifier, 3D shape representation
Citation:
Ajmal Mian, Mohammed Bennamoun, Robyn Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1927-1943, Nov. 2007, doi:10.1109/TPAMI.2007.1105
Usage of this product signifies your acceptance of the Terms of Use.