The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - April (2013 vol.19)
pp: 616-625
ABSTRACT
We present a novel immersive telepresence system that allows distributed groups of users to meet in a shared virtual 3D world. Our approach is based on two coupled projection-based multi-user setups, each providing multiple users with perspectively correct stereoscopic images. At each site the users and their local interaction space are continuously captured using a cluster of registered depth and color cameras. The captured 3D information is transferred to the respective other location, where the remote participants are virtually reconstructed. We explore the use of these virtual user representations in various interaction scenarios in which local and remote users are face-to-face, side-by-side or decoupled. Initial experiments with distributed user groups indicate the mutual understanding of pointing and tracing gestures independent of whether they were performed by local or remote participants. Our users were excited about the new possibilities of jointly exploring a virtual city, where they relied on a world-in-miniature metaphor for mutual awareness of their respective locations.
INDEX TERMS
Calibration, Cameras, Servers, Streaming media, Image reconstruction, Image color analysis, Virtual reality,3D capture., Multi-user virtual reality, telepresence
CITATION
S. Beck, A. Kunert, A. Kulik, B. Froehlich, "Immersive Group-to-Group Telepresence", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 4, pp. 616-625, April 2013, doi:10.1109/TVCG.2013.33
REFERENCES
[1] S. Alers, D. Bloembergen,M. Bügler,D. Hennes,, and K. Tuyls., Mitro: an augmented mobile telepresence robot with assisted control (demonstration). In Proc. of AAMAS2012, pages 1475-1476, Richland, SC, 2012. International Foundation for Autonomous Agents and Multiagent Systems.
[2] ART. Advanced realtime tracking gmbh, 2012. “http://www. ar-tracking.comhome/”.
[3] T. Balogh and P. T. Kovács., Real-time 3d light field transmission. In Proc. of SPIE2010, volume 7724, pages 772406-772406-7, 2010.
[4] H. Benko, R. Jota,, and A. Wilson., Miragetable: freehand interaction on a projected augmented reality tabletop. In Proc. of CHI2012, pages 199-208, New York, NY, USA, 2012. ACM Press.
[5] N. Burrus. Rgbdemo: Demo software to visualize, calibrate and process kinect cameras output, 2012. Available at “http://labs.manctl.comrgbdemo/”.
[6] D. A. Butler, S. Izadi, O. Hilliges, D. Molyneaux, S. Hodges,, and D. Kim., Shake'n'sense: reducing interference for overlapping structured light depth cameras. In Proc. of CHI2012, pages 1933-1936, New York, NY, USA, 2012. ACM Press.
[7] W. Buxton, A. Sellen,, and M. Sheasby., Interfaces for multiparty video-conferences. Video-Mediated Communication, pages 385-400, 1997.
[8] W. A. S. Buxton., Telepresence: integrating shared task and person spaces. In Proc. of Graphics Interface1992, pages 123-129, San Francisco, CA, USA, 1992. Morgan Kaufmann Publishers Inc.
[9] M. Chen., Leveraging the asymmetric sensitivity of eye contact for video-conference. In Proc. of CHI2002, pages 49-56, New York, NY, USA, 2002. ACM Press.
[10] B. Fröhlich,R. Blach, O. Stefani, J. Hochstrate, J. Hoffmann,K. Klüger,, and M. Bues., Implementing multi-viewer stereo displays. In Proc. of WSCG2005, pages 139-146, Plzen, Czech Republic, 2005.
[11] H. Fuchs and U. Neumann., A vision of telepresence for medical consultation and other applications. In Proc. of the 6th Symposium on Robotics Research, pages 565-571, Hidden Valley, PA, USA, 1993. Intl. Foundation for Robotics Research.
[12] S. J. Gibbs, C. Arapis,, and C. J. Breiteneder., Teleport: Towards immersive copresence. Multimedia Systems, 7(3): 214-221, 1999.
[13] M. Gross,S. Würmlin,M. Naef, E. Lamboray, C. Spagno, A. Kunz, E. Koller-Meier, T. Svoboda, L. Van Gool,S. Lang, K. Strehlke, A. V. Moere,, and O. Staadt., blue-c: a spatially immersive display and 3d video portal for telepresence. In Proc. of SIGGRAPH2003, pages 819-827, New York. NY, USA. 2003. ACM Press.
[14] M. Gross,S. Würmlin,M. Naef, E. Lamboray, C. Spagno, A. Kunz, E. Koller-Meier, T. Svoboda, L. Van Gool,S. Lang, K. Strehlke, A. V. Moere,, and O. Staadt., blue-c: a spatially immersive display and 3d video portal for telepresence. ACM Trans. Graph., 22(3): 819-827, July 2003.
[15] H. Ishii and M. Kobayashi., Clearboard: a seamless medium for shared drawing and conversation with eye contact. In Proc. of CHI1992, pages 525-532, New York, NY, USA, 1992. ACM Press.
[16] S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison,, and A. Fitzgibbon., Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In Proc. of UIST2011, pages 559-568, New York, NY, USA, 2011. ACM Press.
[17] A. Jones, M. Lang, G. Fyffe, X. Yu, J. Busch, I. McDowall, M. Bolas,, and P. Debevec., Achieving eye contact in a one-to-many 3d video teleconferencing system. In Proc. of SIGGRAPH2009, pages 64:1-64:8, New York, NY, USA, 2009. ACM Press.
[18] N. P. Jouppi., First steps towards mutually-immersive mobile telepresence. In Proc of CSCW2002, pages 354-363, New York, NY, USA, 2002. ACM Press.
[19] B. Kainz, S. Hauswiesner, G. Reitmayr, M. Steinberger, R. Grasset, L. Gruber, E. Veas, D. Kalkofen, H. Seichter,, and D. Schmalstieg., Omnikinect: real-time dense volumetric data acquisition and applications. In Proc. of VRST2012, VRST '12, pages 25-32, New York, NY, USA, 2012. ACM Press.
[20] K. Kim, J. Bolton, A. Girouard, J. Cooperstock,, and R. Vertegaal., Telehuman: effects of 3d perspective on gaze and pose estimation with a life-size cylindrical telepresence pod. In Proc. of CHI2012, pages 2531-2540, New York, NY, USA, 2012. ACM Press.
[21] R. Kuck, J. Wind, K. Riege,, and M. Bogen., Improving the avango vr/ar framework: Lessons learned. In 5th Workshop of the GI-VR/AR Group, pages 209-220. Shaker, 2008.
[22] A. Kulik, A. Kunert, S. Beck, R. Reichel, R. Blach, A. Zink,, and B. Froehlich., C1×6: a stereoscopic six-user display for co-located collaboration in shared virtual environments. ACM Trans. Graph., 30(6):188: 1-188: 12, Dec. 2011.
[23] G. Kurillo, R. Bajcsy, K. Nahrsted,, and O. Kreylos., Immersive 3d environment for remote collaboration and training of physical activities. In Proc. of VR2008, pages 269-270, Washington, DC, USA, 2008. IEEE Computer Society.
[24] A. Maimone and H. Fuchs., Encumbrance-free telepresence system with real-time 3d capture and display using commodity depth cameras. In Proc. of ISMAR2011, pages 137-146, Washington, DC, USA, 2011. IEEE Computer Society.
[25] A. Maimone and H. Fuchs., A first look at a telepresence system with room-sized real-time 3d capture and large tracked display. In Proc. of ICAT2011, New York, NY, USA, 2011. ACM Press.
[26] A. Maimone and H. Fuchs., Reducing interference between multiple structured light depth sensors using motion. In S. Coquillart,S. Feiner,, and K. Kiyokawa, editors, Proc. of VR2012, pages 51-54, Washington, DC, USA, 2012. IEEE Computer Society.
[27] W. Matusik, and H. Pfister., 3d tv: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Trans. Graph., 23(3): 814-824, Aug. 2004.
[28] P. Milgram and F. Kishino., A taxonomy of mixed reality visual displays. IEICE Trans. Information Systems, E77-D(l2): 1321-1329, 1994. Available at “http://vered.rose.utoronto.ca/people/paul_dir/ IEICE94ieice.html“.
[29] M. Minsky., Telepresence. Omni, 2(6): 45-52, 1980.
[30] R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohli, J. Shotton, S. Hodges,, and A. W. Fitzgibbon., Kinectfusion: Real-time dense surface mapping and tracking. In Proc. of ISMAR2011, pages 127-136, Washington, DC, USA, 2011. IEEE Computer Society.
[31] D. Nguyen and J. Canny., Multiview: spatially faithful group video conferencing. In Proc. of CHI2005, pages 799-808, New York, NY, USA, 2005. ACM Press.
[32] O. Oyekoya, W. Steptoe,, and A. Steed., Sphereavatar: a situated display to represent a remote collaborator. In Proc. of CHI2012, pages 2551-2560, New York, NY, USA, 2012. ACM Press.
[33] F. Pecc, J. Kautz,, and T. Weyrich., Adapting standard video codecs for depth streaming. In Proc. of EGVE2011, pages 59-66, 2011.
[34] B. Petit, J.-D. Lesage, C. Menier, J. Allard, J.-S. Franco, B. Raffin, E. Boyer,, and F. Faure., Multicamera real-time 3d modeling for telepresence and remote collaboration. International Journal of Digital Multimedia Broadcasting, 2010:247 108-12, 2009.
[35] R. Raskar, G. Welch, M. Cutts, A. Lake, L. Stesin,, and H. Fuchs., The office of the future: a unified approach to image-based modeling and spatially immersive displays. In Proc. of SIGGRAPH1998, pages 179-188, New York, NY, USA, 1998. ACM Press.
[36] L. Renambot. Fastdxt: A fast implementation of a dxt compressor.
[37] H. Salzmann, M. Moehring,, and B. Froehlich., Virtual vs. real-world pointing in two-user scenarios. In Proc. of IEEE VR2009, pages 127-130, Washington, DC, USA, 2009. IEEE Computer Society.
[38] O. Schreer, I. Feldmann, N. Atzpadin, P. Eisert, P. Kauff,, and H. Belt., 3 dpresence — a system concept for multi-user and multi-party immersive 3d videoconferencing. In Proc. of CVMP2008, pages 1-8, nov. 2008.
[39] J. P. Springer, C. Sladeczek, M. Scheffler, J. Hochstrate, F. Melchior,, and B. Froehlich., Combining wave field synthesis and multi-viewer stereo displays. In Proc. of VR2006, pages 237-240, Washington, DC, USA, 2006. IEEE Computer Society.
[40] W. Steptoe, J.-M. Normand, W. Oyekoya, F. Pece, G. Giannopoulos, F. Tecchia, A. Steed, T. Weyrich,, and J. Kautz., Acting rehearsal in collaborative multimodal mixed reality environments. Presence — Teleoperators and Virtual Environments, 21(1), 2012.
[41] W. Steptoe, R. Wolff, A. Murgia, E. Guimaraes, J. Rae, P. Sharkey, D. Roberts,, and A. Steed., Eye-tracking for avatar eye-gaze and interactional analysis in immersive collaborative virtual environments. In Proc. of CSCW2008, pages 197-200, New York, NY, USA, 2008. ACM Press.
[42] R. Stoakley, M. J. Conway,, and R. Pausch., Virtual reality on a wim: interactive worlds in miniature. In Proc. of CHI1995, pages 265-272, New York, NY, USA, 1995. ACM Press/Addison-Wesley Publishing Co.
[43] T. Tanikawa, Y. Suzuki, K. Hirota,, and M. Hirose., Real world video avatar: real-time and real-size transmission and presentation of human figure. In Proc. of ICAT2005, pages 112-118, New York, NY, USA, 2005. ACM Press.
[44] J. Tong, J. Zhou, L. Liu, Z. Pan,, and H. Yan., Scanning 3d full human bodies using kinects. IEEE Transactions on Visualization and Computer Graphics, 18(4): 643-650, 2012.
[45] H. Towles, W.-C. Chen, R. Yang, S.-U. Kum, H. Fuchs, J. Mulligan, K. Daniilidis, L. Holden, B. Zeleznik,, and A. Sadagic., 3d telecollaboration over internet2. In International Workshop on Immersive Telepresence, Juan Les Pins, 2002.
[46] R. Vertegaal, I. Weevers, C. Sohn,, and C. Cheung., Gaze-2: conveying eye contact in group video conferencing using eye-controlled camera direction. In Proc. of CHI2003, pages 521-528, New York, NY, USA, 2003. ACM Press.
[47] Z. Zhang., Microsoft kinect sensor and its effect. IEEE Multimedia, 19(2): 4-10, Apr. 2012.
73 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool