The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2013 vol.35)
pp: 1080-1093
O. Aldrian , Dept. of Comput. Sci., Univ. of York, York, UK
W. A. P. Smith , Dept. of Comput. Sci., Univ. of York, York, UK
ABSTRACT
In this paper, we present a complete framework to inverse render faces with a 3D Morphable Model (3DMM). By decomposing the image formation process into geometric and photometric parts, we are able to state the problem as a multilinear system which can be solved accurately and efficiently. As we treat each contribution as independent, the objective function is convex in the parameters and a global solution is guaranteed. We start by recovering 3D shape using a novel algorithm which incorporates generalization error of the model obtained from empirical measurements. We then describe two methods to recover facial texture, diffuse lighting, specular reflectance, and camera properties from a single image. The methods make increasingly weak assumptions and can be solved in a linear fashion. We evaluate our findings on a publicly available database, where we are able to outperform an existing state-of-the-art algorithm. We demonstrate the usability of the recovered parameters in a recognition experiment conducted on the CMU-PIE database.
INDEX TERMS
Lighting, Shape, Harmonic analysis, Solid modeling, Cameras, Rendering (computer graphics), Vectors,texture and illumination analysis, Inverse rendering, face shape
CITATION
O. Aldrian, W. A. P. Smith, "Inverse Rendering of Faces with a 3D Morphable Model", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 5, pp. 1080-1093, May 2013, doi:10.1109/TPAMI.2012.206
REFERENCES
[1] B. Amberg and T. Vetter, "Optimal Landmark Detection Using Shape Models and Branch and Bound," Proc. IEEE Int'l Conf. Computer Vision, pp. 455-462, 2011.
[2] J.J. Atick, P.A. Griffin, and A.N. Redlich, "Statistical Approach to SFS: Reconstruction of 3D Face Surfaces from Single 2D Images," Neural Computation, vol. 8, no. 6, pp. 1321-1340, 1996.
[3] R. Basri and D.W. Jacobs, "Lambertian Reflectance and Linear Subspaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[4] V. Blanz, A. Mehl, T. Vetter, and H.-P. Seidel, "A Statistical Method for Robust 3D Surface Reconstruction from Sparse Data," Proc. Second Int'l Symp. 3D Data Processing, Visualization, and Transmission, pp. 293-300, 2004.
[5] V. Blanz and T. Vetter, "A Morphable Model for the Synthesis of 3D Faces," Proc. ACM Siggraph, pp. 187-194, 1999.
[6] V. Blanz and T. Vetter, "Face Recognition Based on Fitting a 3D Morphable Model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[7] T.F. Cootes, G.J. Edwards, and C.J. Taylor, "Active Appearance Models," Proc. European Conf. Computer Vision, pp. 484-498, 1998.
[8] N.P. Costen, T.F. Cootes, G.J. Edwards, and C.J. Taylor, "Automatic Extraction of the Face Identity-Subspace," Image Vision Computing, vol. 20, pp. 319-329, 2002.
[9] I. Craw and P. Cameron, "Parameterising Images for Recognition and Reconstruction," Proc. British Machine Vision Conf., pp. 367-370, 1991.
[10] C. Creusot, N. Pears, and J. Austin, "3D Landmark Model Discovery from a Registered Set of Organic Shapes," Proc. IEEE CVPR Workshop Point Cloud Processing, 2012.
[11] R.H. Davies, C.J. Twining, T.F. Cootes, J.C. Waterton, and C.J. Taylor, "A Minimum Description Length Approach to Statistical Shape Modelling," IEEE Trans. Medical Imaging, vol. 21, no. 5, pp. 525-537, May 2001.
[12] L. Farkas, Anthropometry of the Head and Face. Raven Press, 1994.
[13] M. Fuchs, V. Blanz, H. Lensch, and H.-P. Seidel, "Reflectance from Images: A Model-Based Approach for Human Faces," IEEE Trans. Visualization and Computer Graphics, vol. 11, no. 3, pp. 296-305, May/June 2005.
[14] A. Georghiades, "Recovering 3-D Shape and Reflectance from a Small Number of Photographs," Proc. Eurographics Symp. Rendering, pp. 230-240, 2003.
[15] A. Georghiades, P. Belhumeur, and D. Kriegman, "From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001.
[16] D.B. Goldman, B. Curless, A. Hertzmann, and S.M. Seitz, "Shape and Spatially-Varying BRDFs from Photometric Stereo," Proc. IEEE Int'l Conf. Computer Vision, 2005.
[17] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[18] A. Hertzmann and S.M. Seitz, "Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1254-1264, Aug. 2005.
[19] T. Heseltine, N. Pears, and J. Austin, "Three-Dimensional Face Recognition Using Surface Space Combinations," Proc. British Machine Vision Conf., 2004.
[20] K. Lee, J. Ho, and D. Kriegman, "Acquiring Linear Subspaces for Face Recognition under Variable Lighting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 684-698, May 2005.
[21] S.R. Marschner and D.P. Greenberg, "Inverse Lighting for Photography," Proc. Fifth Color Imaging Conf., pp. 262-265, 1997.
[22] S. Marschner, S. Westin, E. Lafortune, K. Torrance, and D. Greenberg, "Reflectance Measurements of Human Skin," Technical Report PCG-99-2, Cornell Univ., 1999.
[23] Univ. of Southern California "High-Resolution Light Probe Image Gallery," http://gl.ict.usc.edu/DataHighResProbes, 2011.
[24] P. Paysan, R. Knothe, B. Amberg, S. Romdhani, and T. Vetter, "A 3D Face Model for Pose and Illumination Invariant Face Recognition," Proc. IEEE Int'l Conf. Advanced Video and Signal Based Surveillance, 2009.
[25] R. Ramamoorthi and P. Hanrahan, "A Signal-Processing Framework for Inverse Rendering," Proc. ACM Siggraph, pp. 117-128, 2001.
[26] R. Ramamoorthi, "Modeling Illumination Variation with Spherical Harmonics," Face Processing: Advanced Modeling and Methods, Academic Press, 2005.
[27] R. Ramamoorthi and P. Hanrahan, "A Signal-Processing Framework for Reflection," ACM Trans. Graphics, vol. 23, no. 4, pp. 1004-1042, 2004.
[28] S. Romdhani, V. Blanz, and T. Vetter, "Face Identification by Fitting a 3D Morphable Model Using Linear Shape and Texture Error Functions," Proc. European Conf. Computer Vision, pp. 3-19, 2002.
[29] S. Romdhani, J. Ho, T. Vetter, and D.J. Kriegman, "Face Recognition Using 3-D Models: Pose and Illumination," Proc. IEEE, vol. 94, no. 11, pp. 1977-1999, Nov. 2006.
[30] S. Romdhani and T. Vetter, "Estimating 3D Shape and Texture Using Pixel Intensity, Edges, Specular Highlights, Texture Constraints and a Prior," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 986-993, 2005.
[31] K. Sidorov, S. Richmond, and D. Marshall, "An Efficient Stochastic Approach to Groupwise Non-Rigid Image Registration," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2208-2213, June 2009.
[32] K.A. Sidorov, S. Richmond, and D. Marshall, "Efficient Groupwise Non-Rigid Registration of Textured Surfaces," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2401-2408, June 2011.
[33] T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression Database," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[34] W.A.P. Smith and E.R. Hancock, "Recovering Facial Shape Using a Statistical Model of Surface Normal Direction," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 1914-1930, Dec. 2006.
[35] W.A.P. Smith and E.R. Hancock, "A New Framework for Grayscale and Colour Non-Lambertian Shape-from-Shading," Proc. Asian Conf. Computer Vision, pp. 869-880, 2007.
[36] K. Torrance and E. Sparrow, "Theory for Off-Specular Reflection from Roughened Surfaces," J. Optical Soc. of Am., vol. 57, no. 9, pp. 1105-1114, 1967.
[37] M. Turk and A. Pentland, "Face Recognition Using Eigenfaces," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 586-591, 1991.
[38] L. Zhang and D. Samaras, "Face Recognition from a Single Training Image under Arbitrary Unknown Lighting Using Spherical Harmonics," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 351-363, Mar. 2006.
[39] S. Zhou and R. Chellappa, "Rank Constrained Recognition under Unknown Illuminations," Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, pp. 11-18, 2003.
[40] T. Zickler, S. Mallick, D. Kriegman, and P. Belhumeur, "Color Subspaces as Photometric Invariants," Int'l J. Computer Vision, vol. 79, pp. 13-30, 2008.
27 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool