The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2013 vol.35)
pp: 1039-1050
Yan Cui , Augmented Vision, German Res. Center for Artificial Intell., Kaiserslautern, Germany
S. Schuon , Stylight GmbH, Munich, Germany
S. Thrun , Comput. Sci. Dept., Stanford Univ., Stanford, CA, USA
D. Stricker , Augmented Vision, German Res. Center for Artificial Intell., Kaiserslautern, Germany
C. Theobalt , MPI Inf., Saarbrucken, Germany
ABSTRACT
We describe a method for 3D object scanning by aligning depth scans that were taken from around an object with a Time-of-Flight (ToF) camera. These ToF cameras can measure depth scans at video rate. Due to comparably simple technology, they bear potential for economical production in big volumes. Our easy-to-use, cost-effective scanning solution, which is based on such a sensor, could make 3D scanning technology more accessible to everyday users. The algorithmic challenge we face is that the sensor's level of random noise is substantial and there is a nontrivial systematic bias. In this paper, we show the surprising result that 3D scans of reasonable quality can also be obtained with a sensor of such low data quality. Established filtering and scan alignment techniques from the literature fail to achieve this goal. In contrast, our algorithm is based on a new combination of a 3D superresolution method with a probabilistic scan alignment approach that explicitly takes into account the sensor's noise characteristics.
INDEX TERMS
Cameras, Image resolution, Shape, Image reconstruction, Noise, Solid modeling, Systematics, Kinect, Superresolution, global alignment, rigid transformation, nonrigid transformation, 3D scanning, time-of-flight
CITATION
Yan Cui, S. Schuon, S. Thrun, D. Stricker, C. Theobalt, "Algorithms for 3D Shape Scanning with a Depth Camera", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 5, pp. 1039-1050, May 2013, doi:10.1109/TPAMI.2012.190
REFERENCES
[1] D. Anderson, H. Herman, and A. Kelly, "Experimental Characterization of Commercial Flash Ladar Devices," Proc. Int'l Conf. Sensing Technologies), 2005.
[2] R. Basri and D. Jacobs, "Photometric Stereo with General, Unknown Lighting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 374-381, 2001.
[3] C. Beder, B. Bartczak, and R. Koch, "A Combined Approach for Estimating Patchlets from PMD Depth Images and Stereo Intensity Images," Proc. DAGM Conf. Pattern Recognition, pp. 11-20, 2007.
[4] R. Benjemaa and F. Schmitt, "A Solution for the Registration of Multiple 3D Point Sets Using Unit Quaternions," Proc. European Conf. Computer Vision, pp. 34-50, 1998.
[5] R. Bergevin, M. Soucy, H. Gagnon, and D. Laurendeau, "Towards a General Multi-View Registration Technique," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 5, pp. 540-547, May 1996.
[6] P.J. Besl and N.D. McKay, "A Method for Registration of 3-D Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[7] J.-Y. Bouguet and P. Perona, "3D Photography on Your Desk," Proc. IEEE Int'l Conf. Computer Vision, p. 43, 1998.
[8] B. Brown and S. Rusinkiewicz, "Global Non-Rigid Alignment of 3-D Scans," ACM Trans. Graphics, vol. 26, no. 3,article 21, Aug. 2007.
[9] Y. Cui and D. Hildenbrand, "Pose Estimation Based on Geometric Algebra," Proc. GraVisMa Workshop, 2009.
[10] Y. Cui, S. Schuon, D. Chan, S. Thrun, and C. Theobalt, "3D Shape Scanning with a Time-of-Flight Camera," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1173-1180, 2010.
[11] Y. Cui and D. Stricker, "3D Shape Scanning with a Kinect," Proc. ACM Siggraph, 2011.
[12] B. Curless and M. Levoy, "A Volumetric Method for Building Complex Models from Range Images," Proc. ACM Siggraph, pp. 303-312, 1996.
[13] J.R. Diebel, S. Thrun, and M. Brünig, "A Bayesian Method for Probable Surface Reconstruction and Decimation," ACM Trans. Graphics, vol. 25, no. 1, pp. 39-59, 2006.
[14] M. Erz and B. Jähne, "Radiometric and Spectrometric Calibrations, and Distance Noise Measurement of TOF Cameras," Proc. Third Workshop Dynamic 3-D Imaging, R. Koch and A. Kolb, eds., pp. 28-41, 2009.
[15] Y. Furukawa and J. Ponce., "Accurate Dense, and Robust Multi-View Stereopsis," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[16] U. Hahne and M. Alexa, "Depth Imaging by Combining Time-of-Flight and On-Demand Stereo," Proc. Workshop Dynamic 3D Imaging, pp. 70-83, 2009.
[17] Q.-X. Huang, S. Flöry, N. Gelfand, M. Hofer, and H. Pottmann, "Reassembling Fractured Objects by Geometric Matching," ACM Trans. Graphics, vol. 25, no. 3, pp. 569-578, 2006.
[18] S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison, and A. Fitzgibbon, "Kinectfusion: Real-Time 3D Reconstruction and Interaction Using a Moving Depth Camera," Proc. ACM Symp. User Interface Software and Technology, 2011.
[19] P. Jenke, M. Wand, M. Bokeloh, A. Schilling, and W. Straßer, "Bayesian Point Cloud Reconstruction," Computer Graphics Forum, vol. 25, no. 3 pp 379-388, Sept. 2006.
[20] M. Kazhdan, M. Bolitho, and H. Hoppe, "Poisson Surface Reconstruction," Proc. Fourth Eurographics Symp. Geometry Processing, pp. 61-70, 2006.
[21] Y.J. Kil, B. Medereos, and N. Amenta, "Laser Scanner Superresolution," Proc. Eurographics Symp. Point-Based Graphics, 2006.
[22] Y. Kim, T.C.J. Diebel, J. Kosecka, and B. Mikusic, "Multi-View Image and ToF Sensor Fusion for Dense 3D Reconstruction," Proc. IEEE Workshop 3-D Digital Imaging and Modeling, 2009.
[23] Y. Kim, D. Chan, C. Theobalt, and S. Thrun, "Design and Calibration of a Multi-View ToF Sensor Fusion System," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 1-7, 2008.
[24] A. Kolb, E. Barth, R. Koch, and R. Larsen, "Time-of-Flight Sensors in Computer Graphics," Proc. EUROGRAPHICS Conf., 2009.
[25] K.N. Kutulakos and S.M. Seitz, "A Theory of Shape by Space Carving," technical report, Rochester, N.Y., 1998.
[26] R. Lange, "3D Time-of-Flight Distance Measurement with Custom Solid-State Image Sensors in CMOS/CCD-Technology," PhD thesis, Univ. of Siegen, 2000.
[27] D. Lanman and G. Taubin, "Build Your Own 3D Scanner: 3D Photograhy for Beginners," Proc. ACM Siggraph, pp. 1-87, 2009.
[28] A. Laurentini, "The Visual Hull Concept for Silhouette-Based Image Understanding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 2, pp. 150-162, Feb. 1994.
[29] M. Lindner, I. Schiller, A. Kolb, and R. Koch, "Time-of-Flight Sensor Calibration for Accurate Range Sensing," Computer Vision and Image Understanding, vol. 114, no. 12, pp. 1318-1328, 2010.
[30] W. Matusik, C. Buehler, R. Raskar, S.J. Gortler, and L. McMillan, "Image-Based Visual Hulls," Proc. ACM Siggraph, pp. 369-374, 2000.
[31] Microsoft, http://www.xbox.com/en-USkinect, 2012.
[32] R.M. Murray, Z. Li, and S.S. Sastry, A Mathematical Introduction to Robotic Manipulation, first ed. CRC, Mar. 1994.
[33] A. Myronenko, X. Song, and M. Carreira-Perpinan, "Non-Rigid Point Set Registration: Coherent Point Drift," Proc. Advances Neural Information Processing Systems, vol. 19, p. 1009, 2007.
[34] R. Newcombe, S. Lovegrove, and A. Davison, "DTAM: Dense Tracking and Mapping in Real-Time," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[35] R.A. Newcombe and A.J. Davison, "Live Dense Reconstruction with a Single Moving Camera," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1498-1505, 2010.
[36] A. Rajagopalan, A. Bhavsar, F. Wallhoff, and G. Rigoll, "Resolution Enhancement of PMD Range Maps," Pattern Recognition, pp. 304-313, 2008.
[37] M. Reynolds, J. Doboš, L. Peel, T. Weyrich, and G.J. Brostow, "Capturing Time-of-Flight Data with Confidence," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[38] L. Robert and R. Deriche, "Dense Depth Map Reconstruction: A Minimization and Regularization Approach which Preserves Discontinuities," Proc. European Conf. Computer Vision, 1996.
[39] B. Rosenhahn, "Pose Estimation Revisited," PhD thesis, Univ. of Kiel, Sept. 2003.
[40] S. Rusinkiewicz, O. Hall-Holt, and M. Levoy, "Real-Time 3D Model Acquisition," Proc. ACM Siggraph, pp. 438-446, 2002.
[41] D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," Int'l J. Computer Vision, vol. 47, no. 1, pp. 7-42, 2002.
[42] S. Schuon, C. Theobalt, J. Davis, and S. Thrun, "Lidarboost: Depth Superresolution for TOF 3D Shape Scanning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[43] G. Vogiatzis and C. Hernández, "Practical 3D Reconstruction Based on Photometric Stereo," Computer Vision: Detection, Recognition and Reconstruction, vol. 285, pp. 313-345, 2010.
[44] J. Weickert and H.E. Hagen, Visualization and Processing of Tensor Fields. Springer, 2006.
[45] C. Wu, B. Wilburn, Y. Matsushita, and C. Theobalt, "High-Quality Shape from Multi-View Stereo and Shading under General Illumination," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[46] R. Zhang, P.-S. Tsai, J.E. Cryer, and M. Shah, "Shape from Shading: A Survey," IEEE Tans. Pattern Analysis and Machine Intelligence, vol. 21, no. 8, pp. 690-706, Aug. 1999.
[47] J. Zhu, L. Wang, R. Yang, and J. Davis, "Fusion of Time-of-Flight Depth and Stereo for High Accuracy Depth Maps," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
10 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool