The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - April (2012 vol.18)
pp: 643-650
ABSTRACT
Depth camera such as Microsoft Kinect, is much cheaper than conventional 3D scanning devices, and thus it can be acquired for everyday users easily. However, the depth data captured by Kinect over a certain distance is of extreme low quality. In this paper, we present a novel scanning system for capturing 3D full human body models by using multiple Kinects. To avoid the interference phenomena, we use two Kinects to capture the upper part and lower part of a human body respectively without overlapping region. A third Kinect is used to capture the middle part of the human body from the opposite direction. We propose a practical approach for registering the various body parts of different views under non-rigid deformation. First, a rough mesh template is constructed and used to deform successive frames pairwisely. Second, global alignment is performed to distribute errors in the deformation space, which can solve the loop closure problem efficiently. Misalignment caused by complex occlusion can also be handled reasonably by our global alignment algorithm. The experimental results have shown the efficiency and applicability of our system. Our system obtains impressive results in a few minutes with low price devices, thus is practically useful for generating personalized avatars for everyday users. Our system has been used for 3D human animation and virtual try on, and can further facilitate a range of home-oriented virtual reality (VR) applications.
INDEX TERMS
solid modelling, avatars, computer animation, interactive devices, home-oriented virtual reality applications, 3D full human body model scanning, Microsoft Kinect, 3D scanning devices, depth camera, nonrigid deformation, rough mesh template, successive frame deformation, error distribution, loop closure problem, global alignment algorithm, personalized avatars, 3D human animation, Three dimensional displays, Biological system modeling, Image reconstruction, Shape, Humans, Computational modeling, Geometry, Microsoft Kinect, 3D Body Scanning, global non-igid registration
CITATION
Jing Tong, Jin Zhou, Ligang Liu, Zhigeng Pan, Hao Yan, "Scanning 3D Full Human Bodies Using Kinects", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 4, pp. 643-650, April 2012, doi:10.1109/TVCG.2012.56
REFERENCES
[1] Brett Allen, Brian Curless, and Zoran Popovic, The space of human body shapes: reconstruction and parameterization from range scans. ACM Transactions on Graphics, 22(3) pp. 587-594, 2003.
[2] Edilson de Aguiar, Carsten Stoll, Christian Theobalt, Naveed Ahmed, Hans-Peter Seidel, and Sebastian Thrun, Performance capture from sparse multi-view video. In ACM SIGGRAPH 2008 papers, SIGGRAPH '08, pages 98:1-98:10, New York, NY, USA, 2008. ACM.
[3] Young Min Kim, C. Theobalt, J. Diebel, J. Kosecka, B. Miscusik, and S. Thrun, Multi-view image and tof sensor fusion for dense 3d reconstruction. In IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), pages 1542-1549, 2009.
[4] Microsoft kinect.http://www.xbox.comkinect.2010.
[5] Peter Henry, Michael Krainin, Evan Herbst, Xiaofeng Ren, and Dieter Fox, Rgb-d mapping: Using depth cameras for dense 3d modeling of indoor environments. In International Symposium on Experimental Robotics (ISER), 2010.
[6] Yan Cui and Didier Stricker, 3d shape scanning with a kinect. In ACM SIGGRAPH 2011 Posters, pages 57:1-57:1, New York, NY, USA, 2011. ACM.
[7] Richard A. Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J. Davison, Pushmeet Kohli, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon, Kinectfusion: Real-time dense surface mapping and tracking. In Mixed and Augmented Reality (ISMAR), 2011 10th IEEE International Symposium on, pages 127-136, oct. 2011.
[8] Michael Zollhofer, Michael Martinek, Gnther Greiner, Marc Stamminger, and Jochen Sbmuth, Automatic reconstruction of personalized avatars from 3d face scans. Computer Animation and Virtual Worlds, 22(2-3) pp. 195-202, 2011.
[9] Thibaut Weise, Sofien Bouaziz, Hao Li, and Mark Pauly, Realtime performance-based facial animation. In ACM SIGGRAPH 2011 papers, SIGGRAPH '11, pages 77:1-77:10, New York, NY, USA, 2011. ACM.
[10] Alexander Weiss, David Hirshberg, and Michael J. Black, Home 3d body scans from noisy image and range data. In 13th International Conference on Computer Vision, 2011.
[11] Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis, Scape: shape completion and animation of people. In ACM SIGGRAPH 2005 Papers, SIGGRAPH '05, pages 408-416, New York, NY, USA, 2005. ACM.
[12] Andreas Kolb, Erhardt Barth, Reinhard Koch, Rasmus Larsen, Timeof-flight sensors in computer graphics. M. Pauly, and G. Greiner editors, Eurographics 2009 - State of the Art Reports, pages 119-134. Eurographics, 2009.
[13] Cyberware. http://www.cyberware.com/pricingdomesticPriceList.html 1999.
[14] Niloy J. Mitra, Simon Flöry, Maks Ovsjanikov, Natasha, Gelfand, Leonidas Guibas, and Helmut Pottmann, Dynamic geometry registration. In Proceedings of the fifth Eurographics symposium on Geometry processing, pages 173-182, Aire-la-Ville, Switzerland, Switzerland, 2007. Eurographics Association.
[15] Qi-Xing Huang, Bart Adams, Martin Wicke, and Leonidas J. Guibas, Non-rigid registration under isometric deformations. Computer Graphics Forum, 27(5) pp. 1449-1457, 2008.
[16] Hao Li, Bart Adams, Leonidas J. Guibas, and Mark Pauly, Robust singleview geometry and motion reconstruction. In ACM SIGGRAPH Asia 2009 papers, SIGGRAPH Asia '09, pages 175:1-175:10, New York, NY, USA, 2009. ACM.
[17] Robert W. Sumner, Johannes Schmid, and Mark Pauly, Embedded deformation for shape manipulation. In ACM SIGGRAPH 2007 papers, SIGGRAPH '07, New York, NY, USA, 2007. ACM.
[18] Yuri Pekelny and Craig Gotsman, Articulated object reconstruction and markerless motion capture from depth video. Computer Graphics Forum, 27(2) pp. 399-408, 2008.
[19] Will Chang and Matthias Zwicker, Global registration of dynamic range scans for articulated model reconstruction. ACM Transactions on Graphics, 30:26 pp. 1-2615 May 2011.
[20] E. Baffle, C. Matabosch, and J. Salvi, Overview of 3d registration techniques including loop minimization for the complete acquisition of large manufactured parts and complex environments. In Eight International Conference on Quality Control by Artificial Vision, volume 6356 of Proceedings Of The Society Of Photo-Optical Instrumentation Engineers (SPIE), page 35605. SPIE-INT SOC OPTICAL ENGINEERING, 2007.
[21] Szymon Rusinkiewicz, Benedict Brown, and Michael Kazhdan, 3d scan matching and registration. http://www.cs.princeton.edu/-bjbrowniccv05_course / 2005. ICCV 2005 Short Course.
[22] Paul J. Besl and Neil D. McKay, A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14 pp. 239-256, February 1992.
[23] T. Masuda, K. Sakaue, and N. Yokoya, Registration and integration of multiple range images for 3-d model construction. In Proceedings of the 1996 International Conference on Pattern Recognition (ICPR '96), ICPR '96, pages 879-883, Washington, DC, USA, 1996. IEEE Computer Society.
[24] F. Lu and E. Milios, Globally consistent range scan alignment for environment mapping. Auton. Robots, 4 pp. 333-349, October 1997.
[25] Gregory C. Sharp, Sang W. Lee, and David K. Wehe, Multiview registration of 3d scenes by minimizing error between coordinate frames. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26:10371050, 2004.
[26] Liao Miao, Zhang Qing, Wang Huamin, Yang Ruigang, and Gong Minglun, Modeling deformable objects from a single depth camera. In IEEE 12th International Conference on Computer Vision, pages 167-174, 2009.
[27] O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rossl, and H.-P. Seidel, Laplacian surface editing. Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, 71:175-184, 2004.
[28] Openni. http:/www.openni.org/ 2011.
[29] Multi-camera self-calibration. http://cmp.felk.cvut.cz/~svobodaSelfCal/ 2003.
[30] Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe, Poisson surface reconstruction. In Proceedings of the fourth Eurographics symposium on Geometry processing, SGP '06, pages 61-70, Aire-Ia-Ville, Switzerland, Switzerland, 2006. Eurographics Association.
[31] Nils Hasler, Carsten Stoll, Bodo Rosenhahn, Thorsten Thormahlen, and Hans-Peter Seidel, Estimating body shape of dressed humans. Computers & Graphics, 33(3) pp. 211-216, 2009. IEEE International Conference on Shape Modelling and Applications 2009.
[32] Open source computer vision library (opencv). http://opencv.willowgarage.com/wiki.
[33] Andriy Myronenko, Xubo Song, and Carreira-Perpinn. Non-rigid point set registration: Coherent point drift. In Advances in Neural Information Processing Systems 19 (2007), volume 19, pages 1009-1016. MIT Press, 2007.
[34] Ming Chuang, Linjie Luo, Benedict J. Brown, Szymon Rusinkiewicz, and Michael Kazhdan, Estimating the laplace-beltrami operator by restricting 3d functions. In Proceedings of the Symposium on Geometry Processing, SGP '09, pages 1475-1484, Aire-la-Ville, Switzerland, Switzerland, 2009. Eurographics Association.
[35] Nadia Magnenat-Thalmann, Etienne Lyard, Mustafa Kasap, and Pascal Volino, Adaptive body, motion and cloth. In Motion in Games, volume 5277 of Lecture Notes in Computer Science, pages 63-71. Springer Berlin / Heidelberg, 2008.
[36] David Baraff and Andrew Witkin, Large steps in cloth simulation. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques, SIGGRAPH '98, pages 43-54, New York, NY, USA, 1998. ACM.
[37] Ilya Baran and Jovan Popović, Automatic rigging and animation of 3d characters. In ACM SIGGRAPH 2007 papers, SIGGRAPH '07, New York, NY, USA, 2007. ACM.
[38] Jiejie Zhu, Liang Wang, Jizhou Gao, and Ruigang Yang, Spatial-temporal fusion for high accuracy depth maps using dynamic mrfs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32 pp. 899-909, 2010.
7 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool