This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Automated Mosaics via Topology Inference
March/April 2002 (vol. 22 no. 2)
pp. 44-54

This article presents a complete approach for automated construction of mosaics from images and video, constituting a practical end-to-end system. Local alignment of spatially overlapping frames followed by global consistency provides spatial continuity, while compositing via multiresolution blending provides photometric continuity, so that the mosaic is free of seams that betray the underlying frames. When the camera traverses the scene with a 2D scan pattern, it is necessary to identify and align all spatially overlapping frames, including those not temporally contiguous. To reduce the amount of user interaction and to relax constraints on image capture, the novel topology inference algorithm will automatically determine a graph of spatial overlap relationships within a collection of frames, regardless of the scan pattern. We will highlight the generality of our approach by its application to planar mosaics, spherical mosaics, and geocoded terrestrial mosaics.

1. H.S. Sawhney, R. Kumar, G. Gendel, J. Bergen, D. Dixon, and V. Paragano, “VideoBrushTM: Experiences with Consumer Video Mosaicing,” Proc. IEEE Workshop Applications of Computer Vision, pp. 52-62, 1998.
2. S.E. Chen, "QuickTime VR—An Image-Based Approach to Virtual Environment Navigation," Siggraph 95 Conf. Proc., ACM Press, New York, 1995, pp. 29-38.
3. M. Hodges,"Seeing Data In-Depth," Computer Graphics World, vol. 23, no. 5, May 2000, pp. 43-49.
4. J.W. Berger, M.E. Leventon, N. Hata, W. Wells, and R. Kikinis, “Design Considerations for a Computer-Vision-Enabled Ophthalmic Augmented Reality Environment,” Lecture Notes in Computer Science, vol. 1205, pp. 399-408, 1997.
5. R. Szeliski, “Video Mosaics for Virtual Environments,” IEEE Computer Graphics and Applications, pp. 22-30, Mar. 1996.
6. D. Hockney,Cameraworks, Alfred A. Knopf, New York 1984.
7. H. Sawhney, S. Hsu, and R. Kumar, “Robust Video Mosaicing through Topology Inference and Local to Global Alignment,” Proc. Fifth European Conf. Computer Vision, vol. II, pp. 103-119, 1998.
8. P.J. Burt and E.H. Adelson, “A Multiresolution Spline with Application to Image Mosaics,” ACM Trans. Graphics, vol. 2, no. 4, pp. 217-236, Oct. 1983.
9. J. Shade, S. Gortler, L.-W. He, and R. Szeliski, “Layered Depth Images,” Proc. SIGGRAPH '98, pp. 231-242, 1998.
10. M.M. Oliveira and G. Bishop,"Image-Based Objects," Proc. ACM Symp. Interactive 3D Graphics, ACM Press, New York, 1999, pp. 191-198.
11. R. Szeliski and H.-Y. Shum, “Creating Full View Panoramic Image Mosaics and Environments Maps,” Proc. Computer Graphics, Ann. Conf. Series, vol. 8, pp. 251-258, 1997.
12. R. Wildes et al., "Video Georegistration: Algorithm and Quantitative Evaluation," Proc. Int'l Conf. Computer Vision (ICCV 01), vol. 2,IEEE CS Press, Los Alamitos, Calif., 2001, pp. 343-350.
1. P.R. Wolf,Elements of Photogrammetry, McGraw-Hill, New York, 1983.
2. S.K. Nayar, “Catadioptric Omnidirectional Cameras,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 482-488, June 1997.
3. M. Irani et al., "Efficient Representations of Video Sequences and Their Applications," Signal Processing: Image Comm., vol. 8, no. 4, May 1996, pp. 327-351.
4. R. Szeliski, “Video Mosaics for Virtual Environments,” IEEE Computer Graphics and Applications, pp. 22-30, Mar. 1996.
5. H.S. Sawhney, R. Kumar, G. Gendel, J. Bergen, D. Dixon, and V. Paragano, “VideoBrushTM: Experiences with Consumer Video Mosaicing,” Proc. IEEE Workshop Applications of Computer Vision, pp. 52-62, 1998.
6. E.Y. Kang, I. Cohen, and G.G. Medioni, A Graph-Based Global Registration for 2D Mosaics Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 257-260, 2000.
7. R. Szeliski and H.-Y. Shum, “Creating Full View Panoramic Image Mosaics and Environments Maps,” Proc. Computer Graphics, Ann. Conf. Series, vol. 8, pp. 251-258, 1997.
8. H. Sawhney, S. Hsu, and R. Kumar, “Robust Video Mosaicing through Topology Inference and Local to Global Alignment,” Proc. Fifth European Conf. Computer Vision, vol. II, pp. 103-119, 1998.
9. A. Can, C. Stewart, B. Roysam, and H. Tanenbaum, “A Feature-Based Technique for Joint, Linear Estimation of High-Order Image-to-Mosaic Transformations: Application to Mosaicing the Curved Human Retina,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 585-591, 2000.
10. S. Coorg and S. Teller,"Spherical Mosaics with Quaternions and Dense Correlation," Int'l J. Computer Vision, vol. 37, no. 3, June 2000, pp. 259-273.
11. S. Hsu and H.S. Sawhney, "Influence of Global Constraints and Lens Distortion on Pose and Appearance Recovery From a Purely Rotating Camera," WACV'98, pp. 154-159,Princeton, N.J., Oct. 1998.
12. B. Rousso, S. Peleg, I. Finci, and A. Rav-Acha, “Universal Mosaicing Using Pipe Projection,” Proc. Sixth Int'l Conf. Computer Vision, pp. 945-952, Jan. 1998.
13. S.C. Hsu,"Geocoded Terrestrial Mosaics Using Pose Sensors and Video Registration," Proc. Computer Vision and Pattern Recognition (CVPR 01), vol. 1, IEEE CS Press, Los Alamitos, Calif., 2001, pp. 834-837.
14. J. Davis, Mosaics of Scenes with Moving Objects IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 354-360, June 1998.
15. H.-Y. Shum and R. Szeliski, “Construction and Refinement of Panoramic Mosaics with Global and Local Alignment,” Proc. IEEE Int'l Conf. Computer Vision, pp. 953-958, 1998.
16. H.S. Sawhney et al., "Hybrid Stereo Camera: An IBR Approach for Synthesis of Very High Resolution Stereoscopic Image Sequences," Proc. Siggraph 01, ACM Press, New York, 2001, pp. 451-460.
17. M. Aggarwal and N. Ahuja, “High Dynamic Range Panoramic Imaging,” Proc. Int'l Conf. Computer Vision, vol. I, pp. 2-9, 2001.
18. D. Capel and A. Zisserman, “Automated Mosaicing with Super-Resolution Zoom,” Proc. IEEE Computer Vision and Pattern Recognition, pp. 885-891, 1998.
19. Y.Y. Schechner and S.K. Nayar, “Generalized Mosaicing,” Proc. Int'l Conf. Computer Vision, vol. I, pp. 17-24, 2001.
1. J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hiercharchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, pp. 237-252, 1992.
2. H. Sawhney and R. Kumar, “True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 450-456, 1997.

Index Terms:
environment maps, mosaic construction, image alignment, bundle adjustment, compositing
Citation:
Steve Hsu, Harpreet S. Sawhney, Rakesh Kumar, "Automated Mosaics via Topology Inference," IEEE Computer Graphics and Applications, vol. 22, no. 2, pp. 44-54, March-April 2002, doi:10.1109/38.988746
Usage of this product signifies your acceptance of the Terms of Use.