Issue No.11 - Nov. (2013 vol.19)
pp: 1859-1871
B. H. Le , Dept. of Comput. Sci., Univ. of Houston, Houston, TX, USA
Mingyang Zhu , Nanjing Univ. of Sci. & Technol., Nanjing, China
Zhigang Deng , Dept. of Comput. Sci., Univ. of Houston, Houston, TX, USA
A long-standing problem in marker-based facial motion capture is what are the optimal facial mocap marker layouts. Despite its wide range of potential applications, this problem has not yet been systematically explored to date. This paper describes an approach to compute optimized marker layouts for facial motion acquisition as optimization of characteristic control points from a set of high-resolution, ground-truth facial mesh sequences. Specifically, the thin-shell linear deformation model is imposed onto the example pose reconstruction process via optional hard constraints such as symmetry and multiresolution constraints. Through our experiments and comparisons, we validate the effectiveness, robustness, and accuracy of our approach. Besides guiding minimal yet effective placement of facial mocap markers, we also describe and demonstrate its two selected applications: marker-based facial mesh skinning and multiresolution facial performance capture.
Layout, Face, Deformable models, Optimization, Trajectory, Facial animation,thin-shell deformation, Facial animation, facial deformation, motion capture, marker optimization
B. H. Le, Mingyang Zhu, Zhigang Deng, "Marker Optimization for Facial Motion Acquisition and Deformation", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 11, pp. 1859-1871, Nov. 2013, doi:10.1109/TVCG.2013.84
[1] G. Welch and E. Foxlin, "Motion Tracking: No Silver Bullet, but a Respectable Arsenal," IEEE Computer Graphics and Applications, vol. 22, no. 6, pp. 24-38, Nov./Dec. 2002.
[2] D. Vlasic, R. Adelsberger, G. Vannucci, J. Barnwell, M. Gross, W. Matusik, and J. Popović, "Practical Motion Capture in Everyday Surroundings," ACM Trans. Graphics, vol. 26, article 35, July 2007.
[3] B. Bickel, M. Lang, M. Botsch, M.A. Otaduy, and M. Gross, "Pose-Space Animation and Transfer of Facial Details," Proc. ACM/SIGGRAPH Eurographics Symp. Computer Animation (SCA '08), pp. 57-66, 2008.
[4] W.-C. Ma, A. Jones, J.-Y. Chiang, T. Hawkins, S. Frederiksen, P. Peers, M. Vukovic, M. Ouhyoung, and P. Debevec, "Facial Performance Synthesis Using Deformation-Driven Polynomial Displacement Maps," ACM Trans. Graphics, vol. 27, pp. 121:1-121:10, Dec. 2008.
[5] H. Woltring, "New Possibilities for Human Motion Studies by Real-Time Light Spot Position Measurement," Biotelemetry, vol. 1, pp. 132-146, 1974.
[6] D. Bradley, W. Heidrich, T. Popa, and A. Sheffer, "High Resolution Passive Facial Performance Capture," ACM Trans. Graphics, vol. 29, pp. 41:1-41:10, July 2010.
[7] T. Beeler, F. Hahn, D. Bradley, B. Bickel, P. Beardsley, C. Gotsman, R.W. Sumner, and M. Gross, "High-Quality Passive Facial Performance Capture Using Anchor Frames," ACM Trans. Graphics, vol. 30, pp. 75:1-75:10, Aug. 2011.
[8] L. Zhang, N. Snavely, B. Curless, and S.M. Seitz, "Spacetime Faces: High Resolution Capture for Modeling and Animation," Proc. ACM SIGGRAPH '04 Papers, pp. 548-558, 2004.
[9] E. de Aguiar, C. Theobalt, C. Stoll, and H.-P. Seidel, "Marker-Less Deformable Mesh Tracking for Human Shape and Motion Capture," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '07), June 2007.
[10] B. Bickel, M. Botsch, R. Angst, W. Matusik, M. Otaduy, H. Pfister, and M. Gross, "Multi-Scale Capture of Facial Geometry and Motion," ACM Trans. Graphics, vol. 26, no. 3,article 33, July 2007.
[11] H. Huang, J. Chai, X. Tong, and H.-T. Wu, "Leveraging Motion Capture and 3D Scanning for High-Fidelity Facial Performance Acquisition," Proc. ACM SIGGRAPH '11 Papers, pp. 74:1-74:10, 2011.
[12] F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D.H. Salesin, "Synthesizing Realistic Facial Expressions from Photographs," Proc. ACM SIGGRAPH '98, pp. 75-84, 1998.
[13] T. Weyrich, W. Matusik, H. Pfister, B. Bickel, C. Donner, C. Tu, J. McAndless, J. Lee, A. Ngan, H.W. Jensen, and M. Gross, "Analysis of Human Faces Using a Measurement-Based Skin Reflectance Model," ACM Trans. Graphics, vol. 25, no. 3, pp. 1013-1024, July 2006.
[14] Z. Deng and J.Y. Noh, "Computer Facial Animation: A Survey," Data-Driven 3D Facial Animation, pp. 1-28, Springer, Nov. 2007.
[15] J.-y. Noh and U. Neumann, "Expression Cloning," Proc. ACM SIGGRAPH '01, pp. 277-288, 2001.
[16] R.W. Sumner and J. Popović, "Deformation Transfer for Triangle Meshes," ACM Trans. Graphics, vol. 23, no. 3, pp. 399-405, Aug. 2004.
[17] E. Sifakis, I. Neverov, and R. Fedkiw, "Automatic Determination of Facial Muscle Activations from Sparse Motion Capture Marker Data," Proc. ACM SIGGRAPH '05 Papers, pp. 417-425, 2005.
[18] T. Weise, H. Li, L. van Gool, and M. Pauly, "Face/Off: Live Facial Puppetry," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '09), pp. 7-16, 2009.
[19] T. Weise, S. Bouaziz, H. Li, and M. Pauly, "Realtime Performance-Based Facial Animation," ACM Trans. Graphics, vol. 30, no. 4, pp. 77:1-77:10, Aug. 2011.
[20] Z. Deng, P.-Y. Chiang, P. Fox, and U. Neumann, "Animating Blendshape Faces by Cross-Mapping Motion Capture Data," Proc. Symp. Interactive 3D Graphics and Games (I3D '06), pp. 43-48, 2006.
[21] Y. Seol, J.P. Lewis, J. Seo, B. choi, K. Anjyo, and J. Noh, "Spacetime Expression Cloning for Blendshapes," ACM Trans. Graphics, vol. 30, no. 6, pp. 14:1-14:12, 2011.
[22] P. Joshi, W.C. Tien, M. Desbrun, and F. Pighin, "Learning Controls for Blend Shape Based Realistic Facial Animation," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '03), pp. 187-192, 2003.
[23] M. Meyer and J. Anderson, "Key Point Subspace Acceleration and Soft Caching," ACM Trans. Graphics, vol. 26, article 74, July 2007.
[24] W.-W. Feng, B.-U. Kim, and Y. Yu, "Real-Time Data Driven Deformation Using Kernel Canonical Correlation Analysis," ACM Trans. Graphics, vol. 27, pp. 91:1-91:9, Aug. 2008.
[25] H. Li, T. Weise, and M. Pauly, "Example-Based Facial Rigging," ACM Trans. Graphics, vol. 29, no. 4, pp. 32:1-32:6, July 2010.
[26] J.R. Tena, F. De la Torre, and I. Matthews, "Interactive Region-Based Linear 3D Face Models," Proc. ACM SIGGRAPH '11 Papers, pp. 76:1-76:10, 2011.
[27] C. Bregler, M. Covell, and M. Slaney, "Video Rewrite: Driving Visual Speech with Audio," Proc. ACM SIGGRAPH '97, pp. 353-360, 1997.
[28] M. Brand, "Voice Puppetry," Proc. ACM SIGGRAPH '99, pp. 21-28, 1999.
[29] T. Ezzat, G. Geiger, and T. Poggio, "Trainable Videorealistic Speech Animation," ACM Trans. Graphics, vol. 21, no. 3, pp. 388-398, July 2002.
[30] Z. Deng and U. Neumann, "eFASE: Expressive Facial Animation Synthesis and Editing with Phoneme-Isomap Controls," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '06), pp. 251-260, 2006.
[31] K. Wampler, D. Sasaki, L. Zhang, and Z. Popović, "Dynamic, Expressive Speech Animation from a Single Mesh," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '07), pp. 53-62, 2007.
[32] O. Sorkine and D. Cohen-Or, "Least-Squares Meshes," Proc. Shape Modeling Int'l (SMI '04), pp. 191-199, 2004.
[33] R. Southern and J.J. Zhang, "Motion-Sensitive Anchor Identification of Least-Squares Meshes from Examples," IEEE Trans. Visualization and Computer Graphics, vol. 17, no. 6, pp. 850-856, June 2011.
[34] M. Garland and P.S. Heckbert, "Surface Simplification Using Quadric Error Metrics," Proc. ACM SIGGRAPH '97, pp. 209-216, 1997.
[35] M. Botsch and O. Sorkine, "On Linear Variational Surface Deformation Methods," IEEE Trans. Visualization and Computer Graphics, vol. 14, no. 1, pp. 213-230, Jan. 2008.
[36] M. Meyer, M. Desbrun, P. Schrder, and A. Barr, "Discrete Differential Geometry Operators for Triangulated 2-Manifolds," Proc. Int'l Workshop Visualization and Math., . 2002.
[37] D.P. Bertsekas, Nonlinear Programming, second ed. Athena Scientific, Sept. 1999.
[38] T.A. Davis and W.W. Hager, "Dynamic Supernodes in Sparse Cholesky Update/Downdate and Triangular Solves," ACM Trans. Math. Software, vol. 35, no. 4, pp. 27:1-27:23, Feb. 2009.
[39] Z. Mao, X. Ju, J.P. Siebert, W.P. Cockshott, and A. Ayoub, "Constructing Dense Correspondences for the Analysis of 3D Facial Morphology," Pattern Recognition Letters, vol. 27, no. 6, pp. 597-608, Apr. 2006.
[40] J. Gain and D. Bechmann, "A Survey of Spatial Deformation from a User-Centered Perspective," ACM Trans. Graphics, vol. 27, pp. 107:1-107:21, no. 4, Nov. 2008.
[41] P.G. Kry, D.L. James, and D.K. Pai, "EigenSkin: Real Time Large Deformation Character Skinning in Hardware," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '02), pp. 153-159, 2002.