This Article 
 Bibliographic References 
 Add to: 
Matching 2.5D Face Scans to 3D Models
January 2006 (vol. 28 no. 1)
pp. 31-43
The performance of face recognition systems that use two-dimensional images depends on factors such as lighting and subject's pose. We are developing a face recognition system that utilizes three-dimensional shape information to make the system more robust to arbitrary pose and lighting. For each subject, a 3D face model is constructed by integrating several 2.5D face scans which are captured from different views. 2.5D is a simplified 3D (x, y, z) surface representation that contains at most one depth value (z direction) for every point in the (x, y) plane. Two different modalities provided by the facial scan, namely, shape and texture, are utilized and integrated for face matching. The recognition engine consists of two components, surface matching and appearance-based matching. The surface matching component is based on a modified Iterative Closest Point (ICP) algorithm. The candidate list from the gallery used for appearance matching is dynamically generated based on the output of the surface matching component, which reduces the complexity of the appearance-based matching stage. Three-dimensional models in the gallery are used to synthesize new appearance samples with pose and illumination variations and the synthesized face images are used in discriminant subspace analysis. The weighted sum rule is applied to combine the scores given by the two matching components. Experimental results are given for matching a database of 200 3D face models with 598 2.5D independent test scans acquired under different pose and some lighting and expression changes. These results show the feasibility of the proposed matching scheme.

[1] W. Zhao, R. Chellappa, A. Rosenfeld, and P.J. Phillips, “Face Recognition: A Literature Survey,” VL Technical Report, Univ. of Maryland, Oct. 2000.
[2] Handbook of Face Recognition, S.Z. Li and A.K. Jain, eds. Springer, 2005.
[3] J.Y. Cartoux, J.T. LaPreste, and M. Richetin, “Face Authentication or Recognition by Profile Extraction from Range Images,” Proc. Workshop Interpretation of 3D Scenes, pp. 194-199, 1989.
[4] J. Lee and E. Milios, “Matching Range Images of Human Faces,” Proc. Int'l Conf. Computer Vision, pp. 722-726, 1990.
[5] G. Gordon, “Face Recognition Based on Depth and Curvature Features,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 108-110, 1992.
[6] B. Achermann, X. Jiang, and H. Bunke, “Face Recognition Using Range Images,” Proc. Int'l Conf. Virtual Systems and MultiMedia, pp. 129-136, 1997.
[7] H. Tanaka, M. Ikeda, and H. Chiaki, “Curvature-Based Face Surface Recognition Using Spherical Correlation,” Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 372-377, 1998.
[8] C. Chua, F. Han, and Y. Ho, “3D Human Face Recognition Using Point Signature,” Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 233-238, Mar. 2000.
[9] C. Beumier and M. Acheroy, “Automatic 3D Face Authentication,” Image and Vision Computing, vol. 18, no. 4, pp. 315-321, 2000.
[10] C. Hesher, A. Srivastava, and G. Erlebacher, “PCA of Range Images for Facial Recognition,” Proc. 2002 Int'l Multiconf. Computer Science, 2002.
[11] G. Pan, Z. Wu, and Y. Pan, “Automatic 3D Face Verification from Range Data,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 3, pp. 193-196, 2003.
[12] A.M. Bronstein, M.M. Bronstein, and R. Kimmel, “Expression-Invariant 3D Face Recognition,” Proc. Int'l Conf. Audio- and Video-Based Biometric Person Authentication, pp. 62-70, 2003.
[13] X. Lu, D. Colbry, and A. Jain, “Three-Dimensional Model Based Face Recognition,” Proc. Int'l Conf. Pattern Recognition, pp. 362-366, 2004.
[14] Face Recognition Vendor Test (FRVT), http:/, 2002.
[15] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[16] Cyberware Inc., http:/, 2005.
[17] Minolta Vivid 910 noncontact 3D laser scanner, http://www.minoltausa.comvivid/, 2005.
[18] Proc. Fifth Int'l Conf. 3-D Digital Imaging and Modeling (3DIM), http:/, 2005.
[19] Q. Chen and G. Medioni, “Building 3-D Human Face Models from Two Photographs,” J. VLSI Signal Processing, vol. 27, pp. 127-140, 2001.
[20] Z. Zhang, “Image-Based Modeling of Objects and Human Faces,” Proc. SPIE, vol. 4309, pp. 1-15, Jan. 2001.
[21] B. Moghaddam, J. Lee, H. Pfister, and R. Machiraju, “Model-Based 3D Face Capture with Shape-from-Silhouettes,” Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, pp. 20-27, Oct. 2003.
[22] M. Dimitrijevic, S. Ilic, and P. Fua, “Accurate Face Models from Uncalibrated and Ill-Lit Video Sequences,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 1034-1041, 2004.
[23] K.I. Chang, K.W. Bowyer, and P.J. Flynn, “Multi-Modal 2D and 3D Biometrics for Face Recognition,” Proc. IEEE Workshop Analysis and Modeling of Faces and Gestures, Oct. 2003.
[24] G. Turk and M. Levoy, “Zippered Polygon Meshes from Range Images,” Proc. ACM SIGGRAPH, July 1994.
[25] C. Dorai, G. Wang, A.K. Jain, and C. Mercer, “Registration and Integration of Multiple Object Views for 3D Model Construction,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 83-89, Jan. 1998.
[26] P. Liepa, “Filling Holes in Meshes,” Proc. Eurographics/ACM SIGGRAPH Symp. Geometry Processing, pp. 200-205, 2003.
[27] T. Tasdizen, R. Whitaker, P. Burchard, and S. Osher, “Geometric Surface Smoothing via Anisotropic Diffusion of Normals,” Proc. Visualization '02, 2002.
[28] Geomagic Studio,, 2005.
[29] D.M. Weinstein, “The Analytic 3-D Transform for the Least-Squared Fit of Three Pairs of Corresponding Points,” School of Computing Technical Report, No. UUCS-98-005, Univ. of Utah, Mar. 1998.
[30] C. Dorai and A.K. Jain, “Cosmos— A Representation Scheme for 3D Free-form Objects,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 10, pp. 1115-1130, Oct. 1997.
[31] D. Colbry, X. Lu, A. Jain, and G. Stockman, “3D Face Feature Extraction for Recognition,” Technical Report MSU-CSE-04-39, Computer Science and Eng., Michigan State Univ., East Lansing, Sept. 2004.
[32] R.A. Hummel and S.W. Zucker, “On the Foundations of Relaxation Labeling Processes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 5, no. 3, pp. 267-287, 1983.
[33] P. Besl and N. McKay, “A Method for Registration of 3D Shapes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[34] Y. Chen and G. Medioni, “Object Modeling by Registration of Multiple Range Images,” Image and Vision Computing, vol. 10, no. 3, pp. 145-155, 1992.
[35] Z. Zhang, “Iterative Point Matching for Registration of Free-Form Curves and Surfaces,” Int'l J. Computer Vision, vol. 13, no. 1, pp. 119-152, 1994.
[36] J. Bentley, “Multidimensional Binary Search Trees Used for Associative Searching,” Comm. ACM, vol. 18, no. 9, pp. 509-517, 1975.
[37] J.H. Friedman, J.L. Bentley, and R.A. Finkel, “An Algorithm for Finding Best Matches in Logarithmic Expected Time,” ACM Trans. Math. Software, vol. 3, no. 3, pp. 209-226, 1977.
[38] N. Gelfand, L. Ikemoto, S. Rusinkiewicz, and M. Levoy, “Geometrically Stable Sampling for the ICP Algorithm,” Proc. Int'l Conf. 3D Digital Imaging and Modeling, Oct. 2003.
[39] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, Mar. 1991.
[40] P.N. Belhumeur, J.P. Hespanha, and D.J. Kriegman, “Eigenfaces versus Fisherfaces: Recognition Using Class Specific Linear Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[41] M.S. Bartlett, H.M. Lades, and T.J. Sejnowski, “Independent Component Representations for Face Recognition,” Proc. SPIE, vol. 3299, pp. 528-539, 1998.
[42] S. Shan, Y. Chang, W. Gao, and B. Cao, “Curse of Mis-Alignment in Face Recognition: Problem and a Novel Mis-Alignment Learning Solution,” Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 314-320, 2004.
[43] A.M. Martinez and A.C. Kak, “PCA versus LDA,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 228-233, Feb. 2001.
[44] J. Foley, A. van Dam, S. Feiner, and J. Hughes, Computer Graphics: Principles and Practice, second ed. New York: Addison-Wesley, 1996.
[45] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. New York: Wiley, 2000.
[46] J. Kittler, M. Hatef, R. Duin, and J. Matas, “On Combining Classifiers,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 226-239, Mar. 1998.
[47] K.-P. Li and J.E. Porter, “Normalizations and Selection of Speech Segments for Speaker Recognition Scoring,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 595-597, 1998.
[48] S. Bengio, J. Mariethoz, and S. Marcel, “Evaluation of Biometric Technology on XM2VTS,” technical report, Dalle Molle Inst. for Perceptual Artificial Intelligence, 2001.
[49] USF HumanID 3D Face Dataset, courtesy of Sudeep Sarkar, Univ. of South Florida, Tampa, 2005.
[50] A.K. Jain and A. Ross, “Learning User-Specific Parameters in a Multibiometric System,” Proc. IEEE Int'l Conf. Image Processing, pp. 57-60, 2002.
[51] X. Lu and A. Jain, “Deformation Analysis for 3D Face Matching,” Proc. Seventh IEEE Workshop Applications of Computer Vision, 2005.
[52] I. Matthews and S. Baker, “Active Appearance Models Revisited,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 135-164, 2004.

Index Terms:
Index Terms- Face recognition, 3D model, multimodal, surface matching, appearance-based.
Xiaoguang Lu, Anil K. Jain, Dirk Colbry, "Matching 2.5D Face Scans to 3D Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 31-43, Jan. 2006, doi:10.1109/TPAMI.2006.15
Usage of this product signifies your acceptance of the Terms of Use.