The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - Oct. (2013 vol.19)
pp: 1758-1767
S. Hauswiesner , Inst. for Comput. Graphics & Vision, Graz Univ. of Technol., Graz, Austria
M. Straka , Inst. for Comput. Graphics & Vision, Graz Univ. of Technol., Graz, Austria
G. Reitmayr , Inst. for Comput. Graphics & Vision, Graz Univ. of Technol., Graz, Austria
ABSTRACT
Image-based visual hull rendering is a method for generating depth maps of a desired viewpoint from a set of silhouette images captured by calibrated cameras. It does not compute a view-independent data representation, such as a voxel grid or a mesh, which makes it particularly efficient for dynamic scenes. When users are captured, the scene is usually dynamic, but does not change rapidly because people move smoothly within a subsecond time frame. Exploiting this temporal coherence to avoid redundant calculations is challenging because of the lack of an explicit data representation. This paper analyzes the image-based visual hull algorithm to find intermediate information that stays valid over time and is, therefore, worth to make explicit. We then derive methods that exploit this information to improve the rendering performance. Our methods reduce the execution time by up to 25 percent. When the user's motions are very slow, reductions of up to 50 percent are achieved.
INDEX TERMS
Cameras, Rendering (computer graphics), Coherence, Visualization, Image reconstruction, Surface treatment, Sensors,temporal coherence, Mixed reality, image-based visual hull rendering
CITATION
S. Hauswiesner, M. Straka, G. Reitmayr, "Temporal Coherence in Image-Based Visual Hull Rendering", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 10, pp. 1758-1767, Oct. 2013, doi:10.1109/TVCG.2013.85
REFERENCES
[1] A. Ladikos, S. Benhimane, and N. Navab, "Efficient Visual Hull Computation for Real-Time 3D Reconstruction Using Cuda," Proc. Computer Vision and Pattern Recognition Workshop, pp. 1-8, 2008.
[2] C. Nitschke, A. Nakazawa, and H. Takemura, "Real-Time Space Carving Using Graphics Hardware," IEICE Trans. Information Systems, vol. E90-D, no. 8, pp. 1175-1184, 2007.
[3] D. Knoblauch and F. Kuester, "Region-of-Interest Volumetric Visual Hull Refinement," Proc. 17th ACM Symp. Virtual Reality Software and Technology, pp. 143-150, 2010.
[4] J.-S. Franco and E. Boyer, "Efficient Polyhedral Modeling from Silhouettes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 3, pp. 414-427, Mar. 2009.
[5] W. Matusik, C. Buehler, R. Raskar, S.J. Gortler, and L. McMillan, "Image-Based Visual Hulls," Proc. ACM SIGGRAPH '00, pp. 369-374, 2000.
[6] W. Waizenegger, I. Feldmann, P. Eisert, and P. Kauff, "Parallel High Resolution Real-Time Visual Hull on GPU," ICIP '09: Proc. 16th IEEE Int'l Conf. Image Processing, pp. 4245-4248, 2009.
[7] C. Lee, J. Cho, and K. Oh, "Hardware-Accelerated Jaggy-Free Visual Hulls with Silhouette Maps," VRST '06: Proc. ACM Symp. Virtual Reality Software and Technology, pp. 87-90, 2006.
[8] M. Li, "Towards Real-Time Novel View Synthesis Using Visual Hulls," PhD dissertation, Universität des Saarlandes, 2004.
[9] Y.M. Kim, D. Chan, C. Theobalt, and S. Thrun, "Design and Calibration of a Multi-View TOF Sensor Fusion System," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition Workshops, pp. 1-7, June 2008.
[10] L. Guan, J.-S. Franco, and M. Pollefeys, "3D Object Reconstruction with Heterogeneous Sensor Data," Proc. Int'l Symp. 3D Data Processing, Visualization and Transmission (3DPVT), http://hal.inria.frinria-00349099, 2008.
[11] J. Tong, J. Zhou, L. Liu, Z. Pan, and H. Yan, "Scanning 3D Full Human Bodies Using Kinects," IEEE Trans. Visualization and Computer Graphics, vol. 18, no. 4, pp. 643-650, Apr. 2012.
[12] R.A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A.J. Davison, P. Kohli, J. Shotton, S. Hodges, and A. Fitzgibbon, "Kinectfusion: Real-Time Dense Surface Mapping and Tracking," Proc. 10th IEEE Int'l Symp. Mixed and Augmented Reality (ISMAR '11), pp. 127-136, http://dx.doi.org/10.1109ISMAR. 2011.6092378 , 2011.
[13] A.D. Wilson and H. Benko, "Combining Multiple Depth Cameras and Projectors for Interactions on, above and between Surfaces," Proc. 23nd Ann. ACM Symp. User Interface Software and Technology (UIST '10), pp. 273-282, http://doi.acm.org/10.11451866029.1866073 , 2010.
[14] K. Berger, K. Ruhl, C. Brümmer, Y. Schröder, A. Scholz, and M. Magnor, "Markerless Motion Capture Using Multiple Color-Depth Sensors," Proc. Int'l Workshop Vision, Modeling and Visualization, pp. 317-324, Oct. 2011.
[15] A. Maimone and H. Fuchs, "Reducing Interference between Multiple Structured Light Depth Sensors Using Motion," Proc. IEEE Virtual Reality Short Papers and Posters (VRW), pp. 51-54, http://dblp.uni-trier.de/db/conf/vrvr2012.html#MaimoneF12 , 2012.
[16] A. Butler, S. Izadi, O. Hilliges, D. Molyneaux, S. Hodges, and D. Kim, "Shake'n'sense: Reducing Interference for Overlapping Structured Light Depth Cameras," Proc. SIGCHI Conf. Human Factors in Computing Systems (CHI '12), pp. 1933-1936, 2012.
[17] B. Kainz, S. Hauswiesner, G. Reitmayr, M. Steinberger, R. Grasset, L. Gruber, E. Veas, D. Kalkofen, H. Seichter, and D. Schmalstieg, "Omnikinect: Real-Time Dense Volumetric Data Acquisition and Applications," Proc. Symp. Virtual Reality Software and Technology (VRST), 2012.
[18] C. Kuster, T. Popa, C. Zach, C. Gotsman, and M. Gross, "FreeCam: A Hybrid Camera System for Interactive Free-Viewpoint Video," Proc. Int'l Workshop Vision, Modeling, and Visualization (VMV), 2011.
[19] A. Bogomjakov, C. Gotsman, and M. Magnor, "Free-Viewpoint Video from Depth Cameras," Proc. Int'l Workshop Vision, Modeling, and Visualization (VMV '06), pp. 89-96, Nov. 2006.
[20] J. Gall, C. Stoll, E. de Aguiar, C. Theobalt, B. Rosenhahn, and H.-P. Seidel, "Motion Capture Using Joint Skeleton Tracking and Surface Estimation," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR), 2009.
[21] J. Starck, G. Miller, and A. Hilton, "Video-Based Character Animation," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation, pp. 49-58, http://doi.acm.org/10.11451073368. 1073375 , 2005.
[22] A. Jain, T. Thormählen, H.-P. Seidel, and C. Theobalt, "Moviereshape: Tracking and Reshaping of Humans in Videos," ACM Trans. Graphics, vol. 29, no. 5,article 148, 2010.
[23] F. Xu, Y. Liu, C. Stoll, J. Tompkin, G. Bharaj, Q. Dai, H.-P. Seidel, J. Kautz, and C. Theobalt, "Video-Based Characters: Creating New Human Performances from a Multi-View Video Database," Proc. ACM SIGGRAPH, pp. 32:1-32:10, http://doi.acm.org/10.11451964921.1964927 , 2011.
[24] D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, and J. Davis, "Scape: Shape Completion and Animation of People," ACM Trans. Graphics, vol. 24, no. 3, pp. 408-416, 2005.
[25] M. Simmons and C.H. Séquin, "Tapestry: A Dynamic Mesh-Based Display Representation for Interactive Rendering," Proc. Eurographics Workshop Rendering Techniques, pp. 329-340, 2000.
[26] A. Dayal, C. Woolley, B. Watson, and D. Luebke, "Adaptive Frameless Rendering," Proc. ACM SIGGRAPH, 2005.
[27] B. Walter, G. Drettakis, and D.P. Greenberg, "Enhancing and Optimizing the Render Cache," Proc. 13th Eurographics Workshop Rendering, pp. 37-42, 2002.
[28] D. Nehab, P.V. Sander, J. Lawrence, N. Tatarchuk, and J.R. Isidoro, "Accelerating Real-Time Shading with Reverse Reprojection Caching," Proc. 22nd ACM SIGGRAPH/EUROGRAPHICS Symp. Graphics Hardware, pp. 25-35, 2007.
[29] A. Bigdelou, A. Ladikos, and N. Navab, "Incremental Visual Hull Reconstruction," Proc. British Machine Vision Conf. (BMVC '09), p. 1, 2009.
[30] S. Hauswiesner, M. Straka, and G. Reitmayr, "Coherent Image-Based Rendering of Real-World Objects," Proc. Symp. Interactive 3D Graphics and Games, pp. 183-190, 2011.
[31] M. Straka, S. Hauswiesner, M. Ruether, and H. Bischof, "A Free-Viewpoint Virtual Mirror with Marker-Less User Interaction," Proc. 17th Scandinavian Conf. Image Analysis (SCIA), 2011.
[32] S. Hauswiesner, R. Khlebnikov, M. Steinberger, M. Straka, and D. Schmalstieg, "Multi-GPU Image-Based Visual Hull Rendering," Proc. Eurographics Symp. Parallel Graphics and Visualization (EGPGV), 2012.
30 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool