The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - February (2011 vol.33)
pp: 394-405
Ira Kemelmacher-Shlizerman , University of Washington, Seattle
ABSTRACT
Human faces are remarkably similar in global properties, including size, aspect ratio, and location of main features, but can vary considerably in details across individuals, gender, race, or due to facial expression. We propose a novel method for 3D shape recovery of faces that exploits the similarity of faces. Our method obtains as input a single image and uses a mere single 3D reference model of a different person's face. Classical reconstruction methods from single images, i.e., shape-from-shading, require knowledge of the reflectance properties and lighting as well as depth values for boundary conditions. Recent methods circumvent these requirements by representing input faces as combinations (of hundreds) of stored 3D models. We propose instead to use the input image as a guide to "mold” a single reference model to reach a reconstruction of the sought 3D shape. Our method assumes Lambertian reflectance and uses harmonic representations of lighting. It has been tested on images taken under controlled viewing conditions as well as on uncontrolled images downloaded from the Internet, demonstrating its accuracy and robustness under a variety of imaging conditions and overcoming significant differences in shape between the input and reference individuals including differences in facial expressions, gender, and race.
INDEX TERMS
Computer vision, photometry, shape from shading, 3D reconstruction, lighting, single images, face, depth reconstruction.
CITATION
Ira Kemelmacher-Shlizerman, "3D Face Reconstruction from a Single Image Using a Single Reference Face Shape", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 2, pp. 394-405, February 2011, doi:10.1109/TPAMI.2010.63
REFERENCES
[1] J.J. Atick, P.A. Griffin, and A.N. Redlich, "Statistical Approach to Shape from Shading: Reconstruction of 3D Face Surfaces from Single 2D Images," Neural Computation, vol. 8, no. 6, pp. 1321-1340, 1996.
[2] R. Basri and D.W. Jacobs, "Lambertian Reflectance and Linear Subspaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[3] M.J. Black and P. Anandan, "The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields," Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996.
[4] V. Blanz and T.A. Vetter, "A Morphable Model for the Synthesis of 3d Faces," Proc. ACM SIGGRAPH, vol. I, pp. 187-194, 1999.
[5] M. Castelan, W.A.P. Smith, and E.R. Hancock, "A Coupled Statistical Model for Face Shape Recovery from Brightness Images," IEEE Trans. Image Processing, vol. 16, no. 4, pp. 1139-1151, 2007.
[6] M.K. Chandraker, S. Agarwal, and D.J. Kriegman, "Shadowcuts: Photometric Stereo with Shadows," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[7] T.F. Cootes, G.J. Edwards, and C.J. Taylor, "Active Appearance Models," Proc. European Conf. Computer Vision, 1998.
[8] R. Dovgard and R. Basri, "Statistical Symmetric Shape from Shading for 3d Structure Recovery of Faces," Proc. European Conf. Computer Vision, 2004.
[9] P. Dupuis and J. Oliensis, "An Optimal Control Formulation and Related Numerical Methods for a Problem in Shape Reconstruction," The Annals of Applied Probability, vol. 4, no. 2, pp. 287-346, 1994.
[10] G.J. Edwards, A. Lanitis, C.J. Taylor, and T.F. Cootes, "Modelling the Variability in Face Images," Proc. Second Int'l Conf. Automatic Face and Gesture Recognition, vol. 2, 1996.
[11] D. Frolova, D. Simakov, and R. Basri, "Accuracy of Spherical Harmonic Approximations for Images of Lambertian Objects Under Far and Near Lighting," Proc. European Conf. Computer Vision, pp. 574-587, 2004.
[12] A.S. Georghiades, P.N. Belhumeur, and D.J. Kriegman, "From Few to Many: Illumination Cone Models for Face Recognition Under Variable Lighting and Pose," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001.
[13] C. Hernandez, G. Vogiatzis, and R. Cipolla, "Multi-View Photometric Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp. 548-554, Mar. 2008.
[14] B.K.P. Horn, "Obtaining Shape from Shading Information," The Psychology of Computer Vision, McGraw-Hill, 1975.
[15] Shape from Shading, B.K.P. Horn and M.J. Brooks, eds. MIT Press, 1989.
[16] http://www.csee.usf.edusarkar, USF DARPA Human-ID 3D Face Database, Courtesy of Prof. Sudeep Sarkar, Univ. of South Florida, 2010.
[17] T.M. Hursh, "The Study of Cranial Form: Measurement Techniques and Analytical Methods," The Measures of Man, E. Giles and J. Fiedlaender, eds., Peabody Museum Press, 1976.
[18] S. Katz, A. Tal, and R. Basri, "Direct Visibility of Point Sets," ACM Trans. Graphics, vol. 26, no. 3, pp. 1-11, 2007.
[19] I. Kemelmacher and R. Basri, "Molding Face Shapes by Example," Proc. European Conf. Computer Vision, vol. 1, pp. 277-288, 2006.
[20] R. Kimmel and J.A. Sethian, "Optimal Algorithm for Shape from Shading and Path Planning," J. Math. Imaging and Vision, vol. 14, no. 3, pp. 237-244, 2001.
[21] A. Lanitis, C.J. Taylor, and T.F. Cootes, "Automatic Interpretation and Coding of Face Images Using Flexible Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 743-756, July 1997.
[22] S.R. Marschner, S.H. Westin, E.P.F. Lafortune, K.E. Torrance, and D.P. Greenberg, "Image-Based BRDF Measurement Including Human Skin," Proc. 10th Eurographics Workshop Rendering, pp. 139-152, 1999.
[23] Y. Moses, S. Edelman, and S. Ullman, "Generalization to Novel Images in Upright and Inverted Faces," Perception, vol. 25, pp. 443-461, 1996.
[24] A.P. Pentland, "Finding the Illuminant Direction," J. Optical Soc. of Am., vol. 72, pp. 448-455, 1982.
[25] V. Ramachandran, "Perception of Shape from Shading," Nature, vol. 331, pp. 163-166, 1988.
[26] V.S. Ramachandran, "Visual Perception in People and Machines," AI and the Eye, A. Blake and T. Troscianko, eds., pp. 21-77, J. Wiley & Sons, 1990.
[27] R. Ramamoorthi and P. Hanrahan, "On the Relationship Between Radiance and Irradiance: Determining the Illumination from Images of a Convex Lambertian Object," J. Optical Soc., vol. 18, no. 10, pp. 2448-2459, 2001.
[28] E. Rouy and A. Tourin, "A Viscosity Solutions Approach to Shape-from-Shading," SIAM J. Numerical Analysis, vol. 29, no. 3, pp. 867-884, June 1992.
[29] A. Shashua and T. Riklin-Raviv, "The Quotient Image: Class Based Re-Rendering and Recognition with Varying Illuminations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 129-139, Feb. 2001.
[30] I. Shimshoni, Y. Moses, and M. Lindenbaum, "Shape Reconstruction of 3D Bilaterally Symmetric Surfaces," Int'l J. Computer Vision, vol. 39, no. 2, pp. 97-100, 2000.
[31] T. Sim and T. Kanade, "Combining Models and Exemplars for Face Recognition: An Illuminating Example," Proc. CVPR Workshop Models versus Exemplars, 2001.
[32] W.A.P. Smith and E.R. Hancock, "Recovering Facial Shape and Albedo Using a Statistical Model of Surface Normal Direction," Proc. IEEE Int'l Conf' Computer Vision, 2005.
[33] P.S. Tsai and M. Shah, "Shape from Shading with Variable Albedo," Proc. Optical Eng., pp. 121-1220, Apr. 1998.
[34] L. Zhang and D. Samaras, "Face Recognition Under Variable Lighting Using Harmonic Image Exemplars," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[35] R. Zhang, P.S. Tsai, J.E. Cryer, and M. Shah, "Shape from Shading: A Survey," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 8, pp. 690-706, Aug. 1999.
[36] W. Zhao and R. Chellappa, "Symmetric Shape-from-Shading Using Self-Ratio Image," Int'l J. Computer Vision, vol. 45, pp. 55-75, 2001.
[37] Q. Zheng and R. Chellappa, "Estimation of Illuminant Direction, Albedo, and Shape from Shading," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 680-702, July 1991.
[38] S.K. Zhou, R. Chellappa, and D.W. Jacobs, "Characterization of Human Faces under Illumination Variations Using Rank, Integrability, and Symmetry Constraints," Proc. European Conf. Computer Vision, 2004.
14 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool