• Publication
  • 2011
  • Issue No. 10 - October
  • Abstract - Reconstructing 3D Face Model with Associated Expression Deformation from a Single Face Image via Constructing a Low-Dimensional Expression Deformation Manifold
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Reconstructing 3D Face Model with Associated Expression Deformation from a Single Face Image via Constructing a Low-Dimensional Expression Deformation Manifold
October 2011 (vol. 33 no. 10)
pp. 2115-2121
Shu-Fan Wang, National Tsing Hua University, Hsinchu
Shang-Hong Lai, National Tsing-Hua University, Hsinchu
Facial expression modeling is central to facial expression recognition and expression synthesis for facial animation. In this work, we propose a manifold-based 3D face reconstruction approach to estimating the 3D face model and the associated expression deformation from a single face image. With the proposed robust weighted feature map (RWF), we can obtain the dense correspondences between 3D face models and build a nonlinear 3D expression manifold from a large set of 3D facial expression models. Then a Gaussian mixture model in this manifold is learned to represent the distribution of expression deformation. By combining the merits of morphable neutral face model and the low-dimensional expression manifold, a novel algorithm is developed to reconstruct the 3D face geometry as well as the facial deformation from a single face image in an energy minimization framework. Experimental results on simulated and real images are shown to validate the effectiveness and accuracy of the proposed algorithm.

[1] B. Allen, B. Curless, and Z. Popovic, "The Space of Human Body Shapes: Reconstruction and Parameterization from Range Scans," ACM Trans. Graphics, vol. 22, pp. 587-594, 2003.
[2] B. Guenter, C. Grimm, D. Wood, H. Malvar, and F. Pighin, "Making Faces," ACM Trans. Graphics, pp. 55-66, 1998.
[3] J.Y. Noh and U. Neumann, "Abstract Expression Cloning," Proc. ACM SIGGRAPH, 2001.
[4] L. Zhang, N. Snavely, B. Curless, and S.M. Seitz, "Spacetime Faces: High-Resolution Capture for Modeling and Animation," ACM Trans. Graphics, vol. 23, pp. 548-558, Aug. 2004.
[5] Y. Wang, X. Huang, C.-S. Lee, S. Zhang, Z. Li, D. Samaras, D. Metaxas, A. Elgammal, and P. Huang, "High Resolution Acquisition, Learning and Transfer of Dynamic 3-D Facial Expressions," Proc. EUROGRAPHICS, pp. 677-686, 2004.
[6] Y. Wang, M. Gupta, S. Zhang, S. Wang, X. Gu, D. Samaras, and P. Huang, "High Resolution Tracking of Non-Rigid Motion of Densely Sampled 3D Data Using Harmonic Maps," Int'l J. Computer Vision, vol. 76, no. 3, pp. 283-300, Mar. 2008.
[7] C. Basso, P. Paysan, and T. Vetter, "Registration of Expressions Data Using a 3D Morphable Model," Proc. Seventh Int'l Conf. Automatic Face and Gesture Recognition, pp. 205-210, 2006.
[8] S. Wang, M. Jin, and X.D. Gu, "Conformal Geometry and Its Applications on 3D Shape Matching, Recognition, and Stitching," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 7, pp. 1209-1220, July 2007.
[9] S. Wang, X. Gu, and H. Qin, "Automatic Non-Rigid Registration of 3D Dynamic Data for Facial Expression Synthesis and Transfer," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.
[10] V. Blanz and T. Vetter, "A Morphable Model for the Synthesis of 3D-Faces," SIGGRAPH '99: Proc. 26th Ann. Conf. Computer Graphics and Interactive Techniques, 1999.
[11] V. Blanz and T. Vetter, "Face Recognition Based on Fitting a 3D Morphable Model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[12] Z. Wen and T. Huang, "Capturing Subtle Facial Motions in 3D Face Tracking," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[13] L. Zalewski and S. Gong, "Synthesis and Recognition of Facial Expressions in Virtual 3D Views," Proc. IEEE Sixth Int'l Conf. Automatic Face and Gesture Recognition, 2004.
[14] V. Blanz, C. Basso, T. Poggio, and T. Vetter, "Reanimating Faces in Images and Video," Proc. 24th Ann. Conf. European Assoc. Computer Graphics, 2003.
[15] L. Zhang, Y. Wang, S. Wang, D. Samaras, S. Zhang, and P. Huang, "Image-Driven Re-Targeting and Relighting of Facial Expressions," Proc. Computer Graphics Int'l, 2005.
[16] J. Tenenbaum, V. de Silva, and J. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction," Science, vol. 290, pp. 2319-2323, 2000.
[17] S. Roweis and L. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, vol. 290, pp. 2323-2326, 2000.
[18] S. Roweis, L. Saul, and G. Hinton, "Global Coordination of Local Linear Models," Neural Information Processing Systems, vol. 14, pp. 889-896, 2001.
[19] X. He, D. Cai, S. Yan, and H.-J. Zhang, "Neighborhood Preserving Embedding," Proc. IEEE 10th Int'l Conf. Computer Vision, pp. 1208-1213, 2005.
[20] Y. Chang, C. Hu, and M. Turk, "Manifold of Facial Expression," Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, 2003.
[21] C. Hu, Y. Chang, and M. Turk, "Probabilistic Expression Analysis on Manifolds," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2004.
[22] C. Hu, Y. Chang, R. Feris, and M. Turk, "Manifold Based Analysis of Facial Expression," Proc. IEEE Workshop Face Processing in Video, 2004.
[23] P.J. Besl and H.D. Mckay, "A Method for Registration of 3-D Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[24] R. Pfeifle and H.-P. Seidel, "Triangular B-Splines for Blending and Filling of Polygonal Holes," Proc. Conf. Graphics Interface, pp. 186-193, 1996.
[25] D. Zhang and M. Hebert, "Harmonic Maps and Their Applications in Surface Matching," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, 1999.
[26] U. Pinkall, S.D. Juni, and K. Polthier, "Computing Discrete Minimal Surfaces and Their Conjugates," Experimental Math., vol. 2, pp. 15-36, 1993.
[27] M.X. Nguyen, X. Yuan, and B. Chen, "Geometry Completion and Detail Generation by Texture Synthesis," The Visual Computer, vol. 21, nos. 8-10, pp. 669-678, 2005.
[28] S. Periaswamy and H. Farid, "Elastic Registration in the Presence of Intensity Variations," IEEE Trans. Medical Imaging, vol. 22, no. 7, pp. 865-874, July 2003.
[29] L. Yin, X. Wei, Y. Sun, J. Wang, and M.J. Rosato, "A 3D Facial Expression Database for Facial Behavior Research," Proc. Seventh Int'l Conf. Automatic Face and Gesture Recognition, pp. 211-216, 2006.
[30] R. Basri and D. Jacobs, "Lambertian Reflectance and Linear Subspaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[31] L. Zhang and D. Samaras, "Face Recognition from a Single Training Image under Arbitrary Unknown Lighting Using Spherical Harmonics," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 351-363, Mar. 2006.
[32] K. Levenberg, "A Method for the Solution of Certain Non-Linear Problems in Least Squares," Proc. Quarterly of Applied Math., vol. 2, no. 2, pp. 164-168, 1944.
[33] Y. Wang, L. Zhang, Z. Liu, G. Hua, Z. Wen, Z. Zhang, and D. Samaras, "Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions," IEEE Trans. Pattern Analysis Machine Intelligence, vol. 31, no. 11, pp. 1968-1984, Nov. 2009.
[34] B. Glocker, N. Komodakis, G. Tziritas, N. Navab, and N. Paragios, "Dense Image Registration through MRFs and Efficient Linear Programming," Medical Image Analysis, vol. 12, no. 6, pp. 731-741, 2008.
[35] N. Komodakis, G. Tziritas, and N. Paragios, "Fast, Approximately Optimal Solutions for Single and Dynamic MRFs," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[36] A. Myronenko, X.B. Song, and M.Á. Carreira-Perpiñán, "Non-Rigid Point Set Registration: Coherent Point Drift," Proc. Neural Information Processing Systems, pp. 1009-1016, 2006.
[37] H. Chui and A. Rangarajan, "A New Algorithm for Non-Rigid Point Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 44-51, 2000.
[38] T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression Database," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[39] F. Wallhoff, Facial Expressions and Emotion Database, FG-Net Database, 2006.

Index Terms:
3D face reconstruction, expression modeling, manifold analysis, surface registration.
Citation:
Shu-Fan Wang, Shang-Hong Lai, "Reconstructing 3D Face Model with Associated Expression Deformation from a Single Face Image via Constructing a Low-Dimensional Expression Deformation Manifold," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 2115-2121, Oct. 2011, doi:10.1109/TPAMI.2011.88
Usage of this product signifies your acceptance of the Terms of Use.