This Article 
 Bibliographic References 
 Add to: 
Surface Capture for Performance-Based Animation
May/June 2007 (vol. 27 no. 3)
pp. 21-31
Jonathan Starck, University of Surrey, Guildford, UK
Adrian Hilton, University of Surrey, Guildford, UK
Digital content production traditionally requires highly skilled artists and animators to first manually craft shape and appearance models and then instill the models with a believable performance. Motion capture technology is now increasingly used to record the articulated motion of a real human performance to increase the visual realism in animation. Motion capture is limited to recording only the skeletal motion of the human body and requires the use of specialist suits and markers to track articulated motion. The authors present surface capture, a fully automated system to capture shape and appearance as well as motion from multiple video cameras as a basis to create highly realistic animated content from an actor's performance in full wardrobe. They address wide-baseline scene reconstruction to provide 360-degree appearance from only eight camera views and introduce an efficient scene representation for level of detail control in streaming and rendering. Finally, they demonstrate interactive animation control in a computer games scenario using a captured library of human animation, achieving a frame rate of 300 fps on consumer level graphics hardware.

1. A. Hilton et al., "Whole-Body Modeling of People from Multi-view Images to Populate Virtual Worlds," The Visual Computer: Int'l J. Computer Graphics, vol. 16, no. 7, 2000, pp. 411–436.
2. G. Vogiatzis, P. Torr, and R. Cipolla, "Multi-view Stereo via Volumetric Graph-Cuts," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), vol. 2, IEEE CS Press, 2005, pp. 391–398.
3. J. Starck, G. Miller, and A. Hilton, "Volumetric Stereo with Silhouette and Feature Constraints," Proc. British Machine Vision Conf. (BMVC), vol. 3, British Machine Vision Assoc., 2006, pp. 1189–1198, 2006.
4. Y. Boykov and V. Kolmogorov, "Computing Geodesics and Minimal Surfaces via Graph Cuts," Proc. IEEE Int'l Conf. Computer Vision (ICCV), IEEE CS Press, 2003, pp. 26–33.
5. E. Praun and H. Hoppe, "Spherical Parameterization and Remeshing," ACM Trans. Graphics, 2003, pp. 340–349.
6. K. Zhou, H. Bao, and J. Shi, "3D Surface Filtering Using Spherical Harmonics," Computer-Aided Design, vol. 36, no. 4, 2004, pp. 363–375.
7. D. Steiner and A. Fischer, "Cutting 3D Freeform Objects with Genus-n into Single Boundary Surfaces Using Topological Graphs," Proc. ACM Symp. Solid Modeling and Applications, ACM Press, 2002, pp. 336–343.
8. P. Burt and E. Adelson, "A Multiresolution Spline with Application to Image Mosaics," ACM Trans. Graphics, vol. 2, no. 4, 1983, pp. 217–236.
9. L. Kovar, M. Gleicher, and F. Pighin, "Motion Graphs," ACM Trans. Graphics, 2002, pp. 473–482.
1. T. Kanade, P. Rander, and P. Narayanan, "Virtualized Reality: Constructing Virtual Worlds from Real Scenes," IEEE MultiMedia, vol. 4, no. 1, 1997, pp. 34–47.
2. W. Matusik, C. Buehler, and L. McMillan, "Polyhedral Visual Hulls for Real-time Rendering," Proc. Eurographics Workshop Rendering, Springer-Verlag, 2001, pp. 115–126.
3. J. Carranza et al., "Free-Viewpoint Video of Human Actors," ACM Trans. Graphics, vol. 22, no. 3, 2003, pp. 569–577.
4. O. Grau, "Studio Production System for Dynamic 3D Content," Proc. Int'l Soc. Optical Eng. Visual Comm. and Image Processing, vol. 5150, SPIE, 2003, pp. 80–89.
5. T. Matsuyama et al., "Real-time 3D Shape Reconstruction, Dynamic 3D Mesh Deformation, and High Fidelity Visualization for 3D Video," Computer Vision and Image Understanding, vol. 96, no. 3, 2004, pp. 393–434.
6. S. Vedula, S. Baker, and T. Kanade, "Image-Based Spatio-Temporal Modeling and View Interpolation of Dynamic Events," ACM Trans. Graphics, vol. 24, no. 2, 2005, pp. 240–261.
7. M. Waschbsch et al., "Scalable 3D Video of Dynamic Scenes," The Visual Computer: Int'l J. Computer Graphics, vol. 21, no. 8–10, 2005, pp. 629–638.
8. J. Allard et al., "The Grimage Platform: A Mixed Reality Environment for Interactions," Proc. 4th IEEE Int'l Conf. Computer Vision Systems, IEEE CS Press, 2006, p. 46.
9. C. Zitnick et al., "High-Quality Video View Interpolation Using a Layered Representation," ACM Trans. Graphics (Proc. Siggraph 2004), vol. 23, no. 3, 2004, pp. 600–608.
10. J. Starck and A. Hilton, "Virtual View Synthesis of People from Multiple View Video Sequences," Graphical Models, vol. 67, no. 6, 2005, pp. 600–620.
11. C. Hernandez Esteban and F. Schmitt, "Silhouette and Stereo Fusion for 3D Object Modeling," Computer Vision and Image Understanding, vol. 96, no. 3, 2004, pp. 367–392.
12. Y. Furukawa and J. Ponce, "Carved Visual Hulls for Image-Based Modeling," Proc. European Conf. Computer Vision (ECCV), Springer, 2006, pp. 564–577.
13. J. Starck and A. Hilton, "Model-Based Multiple View Reconstruction of People," Proc. IEEE Int'l Conf. Computer Vision (ICCV), IEEE CS Press, 2003, pp. 915–922.
14. A. Schodl and I. Essa, "Controlled Animation of Video Sprites," Proc. ACM Symp. Computer Animation, 2002, ACM Press, pp. 121–127.
15. J. Starck, G. Miller, and A. Hilton, "Video-Based Character Animation," Proc. ACM Symp. Computer Animation, 2005, ACM Press, pp. 49–58.

Index Terms:
image-based modelling and rendering, video-based character animation
Jonathan Starck, Adrian Hilton, "Surface Capture for Performance-Based Animation," IEEE Computer Graphics and Applications, vol. 27, no. 3, pp. 21-31, May-June 2007, doi:10.1109/MCG.2007.68
Usage of this product signifies your acceptance of the Terms of Use.