This Article 
 Bibliographic References 
 Add to: 
Detail-Preserving Controllable Deformation from Sparse Examples
Aug. 2012 (vol. 18 no. 8)
pp. 1215-1227
Yue Qi, State Key Lab. of Virtual Reality Technol. & Syst., Beihang Univ., Beijing, China
Ling Zhao, State Key Lab. of Virtual Reality Technol. & Syst., Beihang Univ., Beijing, China
KangKang Yin, Dept. of Comput. Sci., Nat. Univ. of Singapore, Singapore, Singapore
Haoda Huang, Microsoft Res. Asia, Mountain View, CA, USA
Yizhou Yu, Univ. of Hong Kong, Hong Kong, China
Xin Tong, Microsoft Res. Asia, Beijing, China
Recent advances in laser scanning technology have made it possible to faithfully scan a real object with tiny geometric details, such as pores and wrinkles. However, a faithful digital model should not only capture static details of the real counterpart but also be able to reproduce the deformed versions of such details. In this paper, we develop a data-driven model that has two components; the first accommodates smooth large-scale deformations and the second captures high-resolution details. Large-scale deformations are based on a nonlinear mapping between sparse control points and bone transformations. A global mapping, however, would fail to synthesize realistic geometries from sparse examples, for highly deformable models with a large range of motion. The key is to train a collection of mappings defined over regions locally in both the geometry and the pose space. Deformable fine-scale details are generated from a second nonlinear mapping between the control points and per-vertex displacements. We apply our modeling scheme to scanned human hand models, scanned face models, face models reconstructed from multiview video sequences, and manually constructed dinosaur models. Experiments show that our deformation models, learned from extremely sparse training data, are effective and robust in synthesizing highly deformable models with rich fine features, for keyframe animation as well as performance-driven animation. We also compare our results with those obtained by alternative techniques.

[1] H. Huang, L. Zhao, K. Yin, Y. Qi, Y. Yu, and X. Tong, "Controllable Hand Deformation from Sparse Examples with Rich Details," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '11), pp. 73-82, 2011.
[2] J.P. Lewis, M. Cordner, and N. Fong, "Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation," Proc. ACM SIGGRAPH '00, pp. 165-172, 2000.
[3] P.-P.J. Sloan, C.F. Rose, and M.F. Cohen, "Shape by Example," Proc. Symp. Interactive 3D Graphics (I3D '01), pp. 135-143, 2001.
[4] A. Mohr and M. Gleicher, "Building Efficient, Accurate Character Skins from Examples," ACM Trans. Graphics, vol. 22, no. 3, pp. 562-568, 2003.
[5] O. Weber, O. Sorkine, Y. Lipman, and C. Gotsman, "Context-Aware Skeletal Shape Deformation," Computer Graphics Forum, vol. 26, no. 3, pp. 265-274, 2007.
[6] R.Y. Wang, K. Pulli, and J. Popović, "Real-Time Enveloping with Rotational Regression," ACM Trans. Graphics, vol. 26, no. 3,article 73, 2007.
[7] P.G. Kry, D.L. James, and D.K. Pai, "EigenSkin: Real Time Large Deformation Character Skinning in Hardware," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '02), pp. 153-159, 2002.
[8] T. Kurihara and N. Miyata, "Modeling Deformable Human Hands from Medical Images," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '04), pp. 355-363, 2004.
[9] T.-Y. Kim and E. Vendrovsky, "DrivenShape: A Data-Driven Approach for Shape Deformation," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '08), pp. 49-55, 2008.
[10] R.W. Sumner, M. Zwicker, C. Gotsman, and J. Popović, "Mesh-Based Inverse Kinematics," ACM Trans. Graphics, vol. 24, no. 3, pp. 488-495, 2005.
[11] K.G. Der, R.W. Sumner, and J. Popović, "Inverse Kinematics for Reduced Deformable Models," ACM Trans. Graphics, vol. 25, no. 3, pp. 1174-1179, 2006.
[12] W.-W. Feng, B.-U. Kim, and Y. Yu, "Real-Time Data Driven Deformation Using Kernel Canonical Correlation Analysis," ACM Trans. Graphics, vol. 27, no. 3,article 91, 2008.
[13] M. Lau, J. Chai, Y.-Q. Xu, and H. Shum, "Face Poser: Interactive Modeling of 3D Facial Expressions Using Model Priors," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '07), pp. 161-170, 2007.
[14] H. Huang, J. Chai, X. Tong, and H.-T. Wu, "Leveraging Motion Capture and 3D Scanning for High-Fidelity Facial Performance Acquisition," ACM Trans. Graphics, vol. 30, no. 4,article 74, 2011.
[15] J.R. Tena, F.D. la Torre, and I. Matthews, "Interactive Region-Based Linear 3D Face Models," ACM Trans. Graphics, vol. 30, no. 4,article 76, 2011.
[16] B. Bickel, M. Botsch, R. Angst, W. Matusik, M. Otaduy, H. Pfister, and M. Gross, "Multi-Scale Capture of Facial Geometry and Motion," ACM Trans. Graphics, vol. 26, no. 3,article 33, 2007.
[17] B. Bickel, M. Lang, M. Botsch, M.A. Otaduy, and M. Gross, "Pose-Space Animation and Transfer of Facial Details," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '08), pp. 57-66, 2008.
[18] W.-C. Ma, A. Jones, J.-Y. Chiang, T. Hawkins, S. Frederiksen, P. Peers, M. Vukovic, M. Ouhyoung, and P. Debevec, "Facial Performance Synthesis Using Deformation-Driven Polynomial Displacement Maps," ACM Trans. Graphics, vol. 27, no. 5,article 121, 2008.
[19] E. de Aguiar, C. Stoll, C. Theobalt, N. Ahmed, H.-P. Seidel, and S. Thrun, "Performance Capture from Sparse Multi-View Video," ACM Trans. Graphics, vol. 27, no. 3,article 98, 2008.
[20] D. Vlasic, I. Baran, W. Matusik, and J. Popović, "Articulated Mesh Animation from Multi-View Silhouettes," ACM Trans. Graphics, vol. 27, no. 3,article 97, 2008.
[21] H. Li, B. Adams, L.J. Guibas, and M. Pauly, "Robust Single-View Geometry and Motion Reconstruction," ACM Trans. Graphics, vol. 28, no. 5, article 175, 2009.
[22] D. Bradley, W. Heidrich, T. Popa, and A. Sheffer, "High Resolution Passive Facial Performance Capture," ACM Trans. Graphics, vol. 29, no. 4,article 41, 2010.
[23] S. Sueda, A. Kaufman, and D.K. Pai, "Musculotendon Simulation for Hand Animation," ACM Trans. Graphics, vol. 27, no. 3,article 83, 2008.
[24] L. Zhang, N. Snavely, B. Curless, and S. Seitz, "Spacetime Faces: High Resolution Capture for Modeling and Animation," ACM Trans. Graphics, vol. 23, no. 3, pp. 546-556, 2004.
[25] A. Golovinskiy, W. Matusik, H. Pfister, S. Rusinkiewicz, and T. Funkhouser, "A Statistical Model for Synthesis of Detailed Facial Geometry," Proc. ACM SIGGRAPH '06, pp. 1025-1034, 2006.
[26] S. Park and J. Hodgins, "Capturing and Animating Skin Deformation in Human Motion," ACM Trans. Graphics, vol. 25, no. 3, pp. 881-889, 2006.
[27] G. Turk, "Re-Tiling Polygonal Surfaces," Proc. ACM SIGGRAPH '92, pp. 55-64, 1992.
[28] R.W. Sumner and J. Popović, "Deformation Transfer for Triangle Meshes," ACM Trans. Graphics, vol. 23, no. 3, pp. 399-405, 2004.
[29] D.L. James and C.D. Twigg, "Skinning Mesh Animations," Proc. ACM SIGGRAPH '05, pp. 399-407, 2005.
[30] W.S. Cleveland and S.J. Devlin, "Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting," J. Am. Statistical Assoc., vol. 83, no. 403, pp. 596-610, 1988.
[31] L.K. Saul and S.T. Roweis, "Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifolds," J. Machine Learning Research, vol. 4, pp. 119-155, 2003.
[32] M. Belkin and P. Niyogi, "Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering," Proc. Advances in Neural Information Processing Systems (NIPS), pp. 585-591, 2001.
[33] L. Kavan, S. Collins, J. Žára, and C. O'Sullivan, "Skinning with Dual Quaternions," Proc. Symp. Interactive 3D Graphics and Games (I3D '07), pp. 39-46, 2007.
[34] S.R. Buss and J.-S. Kim, "Selectively Damped Least Squares for Inverse Kinematics," J. Graphics Tools, vol. 10, no. 3, pp. 37-49, 2005.

Index Terms:
video signal processing,computational geometry,computer animation,feature extraction,image reconstruction,image sequences,solid modelling,performance-driven animation,detail-preserving controllable deformation sparse examples,laser scanning technology,tiny geometric details,object pores,object wrinkles,faithful digital model,static detail capture,data-driven model,smooth large-scale deformation,high-resolution detail capture,nonlinear mapping,sparse control points,bone transformation,global mapping,geometry,pose space,per-vertex displacement,scanned human hand model,scanned face model,face model reconstruction,multiview video sequence,manually constructed dinosaur model,keyframe animation,Deformable models,Face,Training,Bones,Data models,Geometry,Animation,CCA regression.,Detail-preserving deformation,controllable skinning,learning from sparse examples
Yue Qi, Ling Zhao, KangKang Yin, Haoda Huang, Yizhou Yu, Xin Tong, "Detail-Preserving Controllable Deformation from Sparse Examples," IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 8, pp. 1215-1227, Aug. 2012, doi:10.1109/TVCG.2012.88
Usage of this product signifies your acceptance of the Terms of Use.