The Community for Technology Leaders
RSS Icon
Issue No.10 - October (2011 vol.33)
pp: 2115-2121
Shu-Fan Wang , National Tsing Hua University, Hsinchu
Shang-Hong Lai , National Tsing-Hua University, Hsinchu
Facial expression modeling is central to facial expression recognition and expression synthesis for facial animation. In this work, we propose a manifold-based 3D face reconstruction approach to estimating the 3D face model and the associated expression deformation from a single face image. With the proposed robust weighted feature map (RWF), we can obtain the dense correspondences between 3D face models and build a nonlinear 3D expression manifold from a large set of 3D facial expression models. Then a Gaussian mixture model in this manifold is learned to represent the distribution of expression deformation. By combining the merits of morphable neutral face model and the low-dimensional expression manifold, a novel algorithm is developed to reconstruct the 3D face geometry as well as the facial deformation from a single face image in an energy minimization framework. Experimental results on simulated and real images are shown to validate the effectiveness and accuracy of the proposed algorithm.
3D face reconstruction, expression modeling, manifold analysis, surface registration.
Shu-Fan Wang, Shang-Hong Lai, "Reconstructing 3D Face Model with Associated Expression Deformation from a Single Face Image via Constructing a Low-Dimensional Expression Deformation Manifold", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 10, pp. 2115-2121, October 2011, doi:10.1109/TPAMI.2011.88
[1] B. Allen, B. Curless, and Z. Popovic, "The Space of Human Body Shapes: Reconstruction and Parameterization from Range Scans," ACM Trans. Graphics, vol. 22, pp. 587-594, 2003.
[2] B. Guenter, C. Grimm, D. Wood, H. Malvar, and F. Pighin, "Making Faces," ACM Trans. Graphics, pp. 55-66, 1998.
[3] J.Y. Noh and U. Neumann, "Abstract Expression Cloning," Proc. ACM SIGGRAPH, 2001.
[4] L. Zhang, N. Snavely, B. Curless, and S.M. Seitz, "Spacetime Faces: High-Resolution Capture for Modeling and Animation," ACM Trans. Graphics, vol. 23, pp. 548-558, Aug. 2004.
[5] Y. Wang, X. Huang, C.-S. Lee, S. Zhang, Z. Li, D. Samaras, D. Metaxas, A. Elgammal, and P. Huang, "High Resolution Acquisition, Learning and Transfer of Dynamic 3-D Facial Expressions," Proc. EUROGRAPHICS, pp. 677-686, 2004.
[6] Y. Wang, M. Gupta, S. Zhang, S. Wang, X. Gu, D. Samaras, and P. Huang, "High Resolution Tracking of Non-Rigid Motion of Densely Sampled 3D Data Using Harmonic Maps," Int'l J. Computer Vision, vol. 76, no. 3, pp. 283-300, Mar. 2008.
[7] C. Basso, P. Paysan, and T. Vetter, "Registration of Expressions Data Using a 3D Morphable Model," Proc. Seventh Int'l Conf. Automatic Face and Gesture Recognition, pp. 205-210, 2006.
[8] S. Wang, M. Jin, and X.D. Gu, "Conformal Geometry and Its Applications on 3D Shape Matching, Recognition, and Stitching," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 7, pp. 1209-1220, July 2007.
[9] S. Wang, X. Gu, and H. Qin, "Automatic Non-Rigid Registration of 3D Dynamic Data for Facial Expression Synthesis and Transfer," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.
[10] V. Blanz and T. Vetter, "A Morphable Model for the Synthesis of 3D-Faces," SIGGRAPH '99: Proc. 26th Ann. Conf. Computer Graphics and Interactive Techniques, 1999.
[11] V. Blanz and T. Vetter, "Face Recognition Based on Fitting a 3D Morphable Model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[12] Z. Wen and T. Huang, "Capturing Subtle Facial Motions in 3D Face Tracking," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[13] L. Zalewski and S. Gong, "Synthesis and Recognition of Facial Expressions in Virtual 3D Views," Proc. IEEE Sixth Int'l Conf. Automatic Face and Gesture Recognition, 2004.
[14] V. Blanz, C. Basso, T. Poggio, and T. Vetter, "Reanimating Faces in Images and Video," Proc. 24th Ann. Conf. European Assoc. Computer Graphics, 2003.
[15] L. Zhang, Y. Wang, S. Wang, D. Samaras, S. Zhang, and P. Huang, "Image-Driven Re-Targeting and Relighting of Facial Expressions," Proc. Computer Graphics Int'l, 2005.
[16] J. Tenenbaum, V. de Silva, and J. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction," Science, vol. 290, pp. 2319-2323, 2000.
[17] S. Roweis and L. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, vol. 290, pp. 2323-2326, 2000.
[18] S. Roweis, L. Saul, and G. Hinton, "Global Coordination of Local Linear Models," Neural Information Processing Systems, vol. 14, pp. 889-896, 2001.
[19] X. He, D. Cai, S. Yan, and H.-J. Zhang, "Neighborhood Preserving Embedding," Proc. IEEE 10th Int'l Conf. Computer Vision, pp. 1208-1213, 2005.
[20] Y. Chang, C. Hu, and M. Turk, "Manifold of Facial Expression," Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, 2003.
[21] C. Hu, Y. Chang, and M. Turk, "Probabilistic Expression Analysis on Manifolds," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2004.
[22] C. Hu, Y. Chang, R. Feris, and M. Turk, "Manifold Based Analysis of Facial Expression," Proc. IEEE Workshop Face Processing in Video, 2004.
[23] P.J. Besl and H.D. Mckay, "A Method for Registration of 3-D Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[24] R. Pfeifle and H.-P. Seidel, "Triangular B-Splines for Blending and Filling of Polygonal Holes," Proc. Conf. Graphics Interface, pp. 186-193, 1996.
[25] D. Zhang and M. Hebert, "Harmonic Maps and Their Applications in Surface Matching," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, 1999.
[26] U. Pinkall, S.D. Juni, and K. Polthier, "Computing Discrete Minimal Surfaces and Their Conjugates," Experimental Math., vol. 2, pp. 15-36, 1993.
[27] M.X. Nguyen, X. Yuan, and B. Chen, "Geometry Completion and Detail Generation by Texture Synthesis," The Visual Computer, vol. 21, nos. 8-10, pp. 669-678, 2005.
[28] S. Periaswamy and H. Farid, "Elastic Registration in the Presence of Intensity Variations," IEEE Trans. Medical Imaging, vol. 22, no. 7, pp. 865-874, July 2003.
[29] L. Yin, X. Wei, Y. Sun, J. Wang, and M.J. Rosato, "A 3D Facial Expression Database for Facial Behavior Research," Proc. Seventh Int'l Conf. Automatic Face and Gesture Recognition, pp. 211-216, 2006.
[30] R. Basri and D. Jacobs, "Lambertian Reflectance and Linear Subspaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[31] L. Zhang and D. Samaras, "Face Recognition from a Single Training Image under Arbitrary Unknown Lighting Using Spherical Harmonics," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 351-363, Mar. 2006.
[32] K. Levenberg, "A Method for the Solution of Certain Non-Linear Problems in Least Squares," Proc. Quarterly of Applied Math., vol. 2, no. 2, pp. 164-168, 1944.
[33] Y. Wang, L. Zhang, Z. Liu, G. Hua, Z. Wen, Z. Zhang, and D. Samaras, "Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions," IEEE Trans. Pattern Analysis Machine Intelligence, vol. 31, no. 11, pp. 1968-1984, Nov. 2009.
[34] B. Glocker, N. Komodakis, G. Tziritas, N. Navab, and N. Paragios, "Dense Image Registration through MRFs and Efficient Linear Programming," Medical Image Analysis, vol. 12, no. 6, pp. 731-741, 2008.
[35] N. Komodakis, G. Tziritas, and N. Paragios, "Fast, Approximately Optimal Solutions for Single and Dynamic MRFs," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[36] A. Myronenko, X.B. Song, and M.Á. Carreira-Perpiñán, "Non-Rigid Point Set Registration: Coherent Point Drift," Proc. Neural Information Processing Systems, pp. 1009-1016, 2006.
[37] H. Chui and A. Rangarajan, "A New Algorithm for Non-Rigid Point Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 44-51, 2000.
[38] T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression Database," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[39] F. Wallhoff, Facial Expressions and Emotion Database, FG-Net Database, 2006.
36 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool