The Community for Technology Leaders
RSS Icon
Issue No.03 - May/June (2004 vol.24)
pp: 52-63
Antonio Criminisi , Microsoft Research
Sing Bing Kang , Microsoft Research
Simon Winder , Microsoft Research
Richard Szeliski , Microsoft Research
Richard Hartley , Australian National University and National ICT Australia
This article describes an image-based acquisition and rendering system that enables users to explore remote real-world locations. The system is based on a lightweight, high-resolution, multisensor camera that can be used to quickly film a tour through a large environment such as a house or a garden. During the offline authoring phase, the raw video is processed to produce stabilized high-quality, high-dynamic-range video using a combination of novel techniques. These include a stitching algorithm to remove parallax in areas of image overlap, registration and blending algorithms to produce high dynamic range video from alternately exposed video frames, feature tracking to perform camera stabilization, and a video compression scheme that supports selective runtime decompression and random access. The final user experience is enhanced with multimedia elements such as overview maps, video textures, and sound.<p>The user controls the viewpoint and location through a standard game controller, and the resulting user experience feels much like a computer game. The combination of high-resolution, continuous imagery with real-time interactivity provides viewers with an unprecedented sense of presence in interesting real world environments.</p>
Antonio Criminisi, Sing Bing Kang, Simon Winder, Richard Szeliski, Richard Hartley, "Image-Based Interactive Exploration of Real-World Environments", IEEE Computer Graphics and Applications, vol.24, no. 3, pp. 52-63, May/June 2004, doi:10.1109/MCG.2004.1297011
1. F.P. Brooks, "Walkthrough—A Dynamic Graphics System for Simulating Virtual Buildings," Proc. Workshop Interactive 3D Graphics, ACM Press, 1986, pp. 9-21.
2. P.E. Debevec, C.J. Taylor, and J. Malik, "Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-Based Approach," Proc. Siggraph, ACM Press, 1996, pp. 11-20.
3. A. Schödl et al., "Video Textures," Proc. Siggraph, ACM Press, 2000, pp. 489-498.
4. S. Teller et al., "Calibrated, Registered Images of an Extended Urban Area," Int'l J. Computer Vision, vol. 53, no. 1, June 2003, pp. 93-107.
5. E. Chang et al., "Color Filter Array Recovery Using a Threshold-Based Variable Number of Gradients," SPIE vol. 3650, Sensors, Cameras, and Applications for Digital Photography, SPIE, Mar. 1999, pp. 36-43.
6. R. Szeliski and H.-Y. Shum, "Creating Full View Panoramic Image Mosaics and Texture-Mapped Models," Proc. Siggraph 97, ACM Press, 1997, pp. 251-258.
7. M. Uyttendaele et al., High-Quality Image-Based Interactive Exploration of Real-World Environments, tech. report MSR-TR-2003-61, Microsoft Research, Oct. 2003.
8. C. Schmid, R. Mohr, and C. Bauckhage, "Comparing and Evaluating Interest Points," Proc. 6th Int'l Conf. Computer Vision (ICCV 98), IEEE CS Press, 1998, pp. 230-235.
9. C. Buehler, A. Bosse, and L. McMillan, "Non-metric Image-Based Rendering for Video Stabilization," Proc. IEEE Computer Soc. Conf. Computer Vision and Pattern Recognition (CVPR 01), vol. 2, IEEE CS Press, 2001, pp. 609-614.
24 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool