This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Facial Performance Transfer via Deformable Models and Parametric Correspondence
Sept. 2012 (vol. 18 no. 9)
pp. 1511-1519
ASCII Text
x 
 
A. Dhall, M. de la Hunty, A. Asthana, R. Goecke, "Facial Performance Transfer via Deformable Models and Parametric Correspondence," IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 9, pp. 1511-1519, Sept., 2012.
 
 
BibTex
x
 
@article{ 10.1109/TVCG.2011.157,
author = {A. Dhall and M. de la Hunty and A. Asthana and R. Goecke},
title = {Facial Performance Transfer via Deformable Models and Parametric Correspondence},
journal ={IEEE Transactions on Visualization and Computer Graphics},
volume = {18},
number = {9},
issn = {1077-2626},
year = {2012},
pages = {1511-1519},
doi = {http://doi.ieeecomputersociety.org/10.1109/TVCG.2011.157},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Visualization and Computer Graphics
TI - Facial Performance Transfer via Deformable Models and Parametric Correspondence
IS - 9
SN - 1077-2626
SP1511
EP1519
EPD - 1511-1519
A1 - A. Dhall,
A1 - M. de la Hunty,
A1 - A. Asthana,
A1 - R. Goecke,
PY - 2012
KW - solid modelling
KW - cinematography
KW - image texture
KW - regression analysis
KW - movie dubbing industry
KW - parametric correspondence
KW - computer graphics community
KW - active appearance model
KW - deformable face model-based approaches
KW - computer vision
KW - nonrigid shape
KW - nonrigid texture
KW - sparse linear regression method
KW - cross-language facial performance transfer
KW - Face
KW - Shape
KW - Computational modeling
KW - Active appearance model
KW - Training
KW - Deformable models
KW - Solid modeling
KW - face modeling and animation.
KW - Active appearance models
KW - facial performance transfer
VL - 18
JA - IEEE Transactions on Visualization and Computer Graphics
ER -
 
A. Dhall, RSISE, Australian Nat. Univ., Canberra, ACT, Australia
M. de la Hunty, Australian Nat. Univ., Canberra, ACT, Australia
A. Asthana, RSISE, Australian Nat. Univ., Canberra, ACT, Australia
R. Goecke, Fac. of Inf. Sci. & Eng., Univ. of Canberra, Canberra, ACT, Australia
The issue of transferring facial performance from one person's face to another's has been an area of interest for the movie industry and the computer graphics community for quite some time. In recent years, deformable face models, such as the Active Appearance Model (AAM), have made it possible to track and synthesize faces in real time. Not surprisingly, deformable face model-based approaches for facial performance transfer have gained tremendous interest in the computer vision and graphics community. In this paper, we focus on the problem of real-time facial performance transfer using the AAM framework. We propose a novel approach of learning the mapping between the parameters of two completely independent AAMs, using them to facilitate the facial performance transfer in a more realistic manner than previous approaches. The main advantage of modeling this parametric correspondence is that it allows a "meaningful” transfer of both the nonrigid shape and texture across faces irrespective of the speakers' gender, shape, and size of the faces, and illumination conditions. We explore linear and nonlinear methods for modeling the parametric correspondence between the AAMs and show that the sparse linear regression method performs the best. Moreover, we show the utility of the proposed framework for a cross-language facial performance transfer that is an area of interest for the movie dubbing industry.

[1] F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. Salesin, "Synthesizing Realistic Facial Expressions from Photographs," Proc. ACM SIGGRAPH, pp. 75-84, 1998.
[2] J. Noh and U. Neumann, "Expression Cloning," Proc. ACM SIGGRAPH, pp. 277-288, 2001.
[3] Z. Liu, Y. Shan, and Z. Zhang, "Expressive Expression Mapping with Ratio Images," Proc. ACM SIGGRAPH, pp. 271-276, 2001.
[4] V. Blanz, C. Basso, T. Vetter, and T. Poggio, "Reanimating Faces in Images and Video," Proc. EUROGRAPHICS, pp. 641-650, 2003.
[5] Y.-J. Chang and T. Ezzat, "Transferable Videorealistic Speech Animation," Proc. ACM SIGGRAPH, pp. 143-151, 2005.
[6] B. Theobald, I. Matthews, J. Cohn, and S. Boker, "Real-Time Expression Cloning Using Appearance Models," Proc. Ninth Int'l Conf. Multimodal Interfaces (ICMI '07), pp. 134-139, 2007.
[7] M.J. Jones and T. Poggio, "Multidimensional Morphable Models: A Framework for Representing and Matching Object Classes," Int'l J. Computer Vision, vol. 29, no. 2, pp. 107-131, 1998.
[8] T. Leyvand, D. Cohen-Or, G. Dror, and D. Lischinski, "Data-Driven Enhancement of Facial Attractiveness," ACM Trans. Graphics, vol. 27, no. 3, 2008, doi: 10.1145/1399504.1360637.
[9] D. Vlasic, M. Brand, H. Pfister, and J. Popović, "Face Transfer with Multilinear Models," ACM Trans. Graphics, vol. 24, no. 3, pp. 426-433, 2005.
[10] G. Edwards, C. Taylor, and T. Cootes, "Interpreting Face Images Using Active Appearance Models," Proc. Third IEEE Int'l Conf. Automatic Face and Gesture Recognition (FG '98), pp. 300-305, 1998.
[11] R. Goecke and B. Millar, "The Audio-Video Australian English Speech Data Corpus AVOZES," Proc. Int'l Conf. Spoken Language Processing (ICSLP '04), pp. 2525-2528, 2004.
[12] S. Baker, R. Gross, and I. Matthews, "Lucas-Kanade 20 Years on: A Unifying Framework: Part 3," Technical Report CMU-RITR-03-35, Robotics Inst., Carnegie Mellon Univ., 2003.
[13] S.-J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, "An Interior-Point Method for Large-Scale L1-Regularized Least Squares," IEEE J. Selected Topics in Signal Processing, vol. 1, no. 4, pp. 606-617, Dec. 2007.
[14] D.L. Donoho, "For Most Large Underdetermined Systems of Linear Equations, the Minimal L1-Norm Solution is also the Sparsest Solution," Comm. Pure and Applied Math., vol. 59, pp. 797-829, 2004.
[15] C.E. Rasmussen, Gaussian Processes for Machine Learning. MIT Press, 2006.
[16] P. Boyle and M. Frean, "Dependent Gaussian Processes," Proc. Advances in Neural Information Processing Systems (NIPS 17), pp. 217-224, 2005.
[17] M. Alvarez and N.D. Lawrence, "Sparse Convolved Gaussian Processes for Multi-Output Regression," Proc. Advances in Neural Information Processing Systems (NIPS 21), pp. 57-64, 2008.
[18] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004.
[19] R.W. Sumner and J. Popovic, "Deformation Transfer for Triangle Meshes," Proc. ACM SIGGRAPH, pp. 399-405, 2004.
[20] Z. Wang, A.C. Bovik, H.R. Sheikh, S. Member, E.P. Simoncelli, and S. Member, "Image Quality Assessment: From Error Visibility to Structural Similarity," IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.
[21] A. Dhall, A. Asthana, and R. Goecke, "Facial Expression Based Automatic Album Creation," Proc. Seventh Int'l Conf. Neural Information Processing (ICONIP '10), pp. 485-492, 2010.
[22] A. Dhall, A. Asthana, and R. Goecke, "A SSIM Based Approach for Finding Similar Facial Expressions," Proc. IEEE Int'l Conf. Automated Face and Gesture Recognition and Workshops (FG '11), 2011.
[23] T. Jan and A. Zaknich, "An Adjustable Model for Linear to Nonlinear Regression," Proc. IEEE Int'l Joint Conf. Neural Networks, pp. 846-850, 1999.
[24] P. Vacha, "Texture Similarity Measure," Proc. Contributed Papers: Part I—Math. and Computer Sciences (WDS '05), pp. 47-52, 2005.

Index Terms:
solid modelling,cinematography,image texture,regression analysis,movie dubbing industry,parametric correspondence,computer graphics community,active appearance model,deformable face model-based approaches,computer vision,nonrigid shape,nonrigid texture,sparse linear regression method,cross-language facial performance transfer,Face,Shape,Computational modeling,Active appearance model,Training,Deformable models,Solid modeling,face modeling and animation.,Active appearance models,facial performance transfer
Citation:
A. Dhall, M. de la Hunty, A. Asthana, R. Goecke, "Facial Performance Transfer via Deformable Models and Parametric Correspondence," IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 9, pp. 1511-1519, Sept. 2012, doi:10.1109/TVCG.2011.157
Usage of this product signifies your acceptance of the Terms of Use.