This Article 
 Bibliographic References 
 Add to: 
Effective Replays and Summarization of Virtual Experiences
April 2012 (vol. 18 no. 4)
pp. 607-616
K. Ponto, Dept. of Comput. Sci., Univ. of Wisconsin, Madison, WI, USA
J. Kohlmann, Dept. of Comput. Sci., Univ. of Wisconsin, Madison, WI, USA
M. Gleicher, Dept. of Comput. Sci., Univ. of Wisconsin, Madison, WI, USA
Direct replay of the experience of a user in a virtual environment is difficult for others to watch due to unnatural camera motions. We present methods for replaying and summarizing these egocentric experiences that effectively communicate the user's observations while reducing unwanted camera movements. Our approach summarizes the viewpoint path as a concise sequence of viewpoints that cover the same parts of the scene. The core of our approach is a novel content-dependent metric that can be used to identify similarities between viewpoints. This enables viewpoints to be grouped by similar contextual view information and provides a means to generate novel viewpoints that can encapsulate a series of views. These resulting encapsulated viewpoints are used to synthesize new camera paths that convey the content of the original viewer's experience. Projecting the initial movement of the user back on the scene can be used to convey the details of their observations, and the extracted viewpoints can serve as bookmarks for control or analysis. Finally we present performance analysis along with two forms of validation to test whether the extracted viewpoints are representative of the viewer's original observations and to test for the overall effectiveness of the presented replay methods.

[1] M. Alexa, Linear combination of transformations. ACM Trans. Graph., 21 pp. 380-387, July 2002.
[2] C. Andújar, P. Vázquez, and M. Fairén, Way-finder: guided tours through complex walkthrough models. In Computer Graphics Forum, volume 23, pages 499-508. Wiley Online Library, 2004.
[3] C. Buehler, M. Bosse, and L. McMillan, Non-metric image-based rendering for video stabilization. In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, volume 2, pages 11-609 - 11-614 vol. 2, 2001.
[4] D. Burns and R. Osfield, Open scene graph a: Introduction, b: Examples and applications. In Proceedings of the IEEE Virtual Reality 2004, page 265. IEEE Computer Society, 2004.
[5] N. Burtnyk, A. Khan, G. Fitzmaurice, R. Balakrishnan, and G. Kurtenbach, Stylecam: interactive stylized 3d navigation using integrated spatial & temporal controls. In Proceedings of the 15th annual ACM symposium on User interface software and technology, UIST '02, pages 101-110, New York, NY, USA, 2002. ACM.
[6] N. Burtnyk, A. Khan, G. Fitzmaurice, and G. Kurtenbach, Showmotion: camera motion based 3d design review. In Proceedings of the 2006 symposium on Interactive 3D graphics and games, I3D '06, pages 167-174, New York, NY, USA, 2006. ACM.
[7] M. Christie, P. Olivier, and J.-M. Normand, Camera control in computer graphics. Computer Graphics Forum, 27(8) 2197-2218 2008.
[8] C. Cyr and B. Kimia, A similarity-based aspect-graph approach to 3d object recognition. International Journal of Computer Vision, 57(1) 5-22, 2004.
[9] J. S. Downs, M. B. Holbrook, S. Sheng, and L. F. Cranor, Are your participants gaming the system?: screening mechanical turk workers. In Proceedings of the 28th international conference on Human factors in computing systems, CHI ' 10, pages 2399-2402, New York, NY, USA, 2010. ACM.
[10] S. Fleishman, D. Cohen-Or, and D. Lischinski. Automatic, camera placement for image-based modeling. Computer Graphics Forum, 19(2) 101-110, 2000.
[11] M.L. Gleicher and F. Liu, Re-cinematography: Improving the camera-work of casual video. ACM Trans. Multimedia Comput. Commun. Appl., 5 2:1-2:28, October 2008.
[12] V. GOVINDU, Lie-algebraic averaging for globally consistent motion estimation. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004.
[13] A. Hawkins and C. M. Grimm, Camera keyframing using linear interpolation of matrices. journal of graphics, gpu, and game tools, 12(3) pp. 55-69, 2007.
[14] Z. He and C. Zhao, A recording method of virtual reality interaction. In Communication Software and Networks, 2009. ICCSN '09. International Conference on, pages 666-669, feb. 2009.
[15] L. Kavan, S. Collins, C. OSullivan, and J. Zara, Dual quaternions for rigid transformation blending. Technical report TCDCS200646 Trinity College Dublin, 2009.
[16] E. Kilgariff and R. Fernando, The geforce 6 series gpu architecture. In ACM SIGGRAPH 2005 Courses, SIGGRAPH '05, New York, NY, USA, 2005. ACM.
[17] A. Kittur, E. H. Chi, and B. Suh. Crowdsourcing, user studies with mechanical turk. In Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, CHI '08, pages 453-456, New York, NY, USA, 2008. ACM.
[18] L. Kozlowski and K. Bryant, Sense of direction, spatial orientation, and cognitive maps. Journal of Experimental Psychology: Human Perception and Performance, 3(4): 590, 1977.
[19] E. Krahmer and I. vander Sluis, A new model for generating multimodal referring expressions. In Proceedings of 9th European Workshop on Natural Language Generation (ENLG-2003), pages 47-54. Citeseer, 2003.
[20] A. Lablack, F. Maquet, N. Ihaddadene, and C. Djeraba, Visual gaze projection in front of a target scene. In Multimedia and Expo, 2009. ICME 2009. IEEE International Conference on, pages 1839-1840, 28 2009-july 3 2009.
[21] F. Liu, M. Gleicher, H. Jin, and A. Agarwala, Content-preserving warps for 3d video stabilization. ACM Transcations on Graphics (Proceedings of ACM SIGGRAPH 2009), 28(3), 2009.
[22] K. Mühler, M. Neugebauer, C. Tietjen, and B. Preim, Viewpoint selection for intervention planning. In IEEE/eurographics symposium on visualization (EuroVis), pages 267-274. Citeseer, 2007.
[23] A. Murgia, R. Wolff, W. Steptoe, P. Sharkey, D. Roberts, E. Guimaraes, A. Steed, and J. Rae, A tool for replay and analysis of gaze-enhanced multiparty sessions captured in immersive collaborative environments. In Proceedings of the 2008 12th IEEE/ACM International Symposium on Distributed Simulation and Real-Time Applications, DS-RT '08, pages 252-258, Washington, DC, USA, 2008. IEEE Computer Society.
[24] R. O'Brien and M. Kaiser, Manova method for analyzing repeated measures designs: an extensive primer. Psychological Bulletin, 97(2) p. 316, 1985.
[25] J.D. Owens, D. Luebke, N. Govindaraju, M. Harris, J. Krger, A. E. Lefohn, and T. J. Purcell, A survey of general-purpose computation on graphics hardware. Computer Graphics Forum, 26(1) pp. 80-113, 2007.
[26] M. Segal, C. Korobkin, R. van Widenfelt, J. Foran, and P. Haeberli, Fast shadows and lighting effects using texture mapping. SIGGRAPH Comput. Graph., 26:249-252, July 1992.
[27] C. Shannon and W. Weaver, The mathematical theory of communication. Citeseer, 1959.
[28] T. Sun, J. Karlsson, W. Peng, P. Wall, and Z. Langford, Managing interactions in the collaborative 3d docuspace for enterprise applications. In Proceedings of the International Conference on Human-centric Computing 2011 and Embedded and Multimedia Computing 2011: HumanCom and EMC 2011, volume 102, page 135. Springer, 2011.
[29] B. T. Truong and S. Venkatesh, Video abstraction: A systematic review and classification. ACM Trans. Multimedia Comput. Commun. Appl.,3 February 2007.
[30] M. Tsang, G. W. Fitzmaurice, G. Kurtenbach, A. Khan, and B. Buxton, Boom chameleon: simultaneous capture of 3d viewpoint, voice and gesture annotations on a spatially-aware display. In Proceedings of the 15th annual ACM symposium on User interface software and technology, UIST '02, pages 111-120, New York, NY, USA, 2002. ACM.
[31] E.A. Wernert and A. J. Hanson, A framework for assisted exploration with collaboration. Visualization Conference, IEEE, 0:39, 1999.
[32] L. Williams, Casting curved shadows on curved surfaces. SIGGRAPH Comput. Graph., 12 pp. 270-274, August 1978.

Index Terms:
virtual reality,cameras,performance analysis,virtual experience replay,virtual experience summarization,virtual environment,unnatural camera motions,egocentric experiences,user observations,viewpoint path summarization,content-dependent metric,contextual view information,viewpoint extraction,Cameras,Measurement,Graphics processing unit,Equations,Three dimensional displays,Geometry,Virtual environments,Bookmarking.,Virtual Reality,Viewpoint Similarity,Summarization,GPU
K. Ponto, J. Kohlmann, M. Gleicher, "Effective Replays and Summarization of Virtual Experiences," IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 4, pp. 607-616, April 2012, doi:10.1109/TVCG.2012.41
Usage of this product signifies your acceptance of the Terms of Use.