This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Stereo Reconstruction from Multiperspective Panoramas
January 2004 (vol. 26 no. 1)
pp. 45-62

Abstract—A new approach to computing a panoramic (360 degrees) depth map is presented in this paper. Our approach uses a large collection of images taken by a camera whose motion has been constrained to planar concentric circles. We resample regular perspective images to produce a set of multiperspective panoramas and then compute depth maps directly from these resampled panoramas. Our panoramas sample uniformly in three dimensions: rotation angle, inverse radial distance, and vertical elevation. The use of multiperspective panoramas eliminates the limited overlap present in the original input images and, thus, problems as in conventional multibaseline stereo can be avoided. Our approach differs from stereo matching of single-perspective panoramic images taken from different locations, where the epipolar constraints are sine curves. For our multiperspective panoramas, the epipolar geometry, to the first order approximation, consists of horizontal lines. Therefore, any traditional stereo algorithm can be applied to multiperspective panoramas with little modification. In this paper, we describe two reconstruction algorithms. The first is a cylinder sweep algorithm that uses a small number of resampled multiperspective panoramas to obtain dense 3D reconstruction. The second algorithm, in contrast, uses a large number of multiperspective panoramas and takes advantage of the approximate horizontal epipolar geometry inherent in multiperspective panoramas. It comprises a novel and efficient 1D multibaseline matching technique, followed by tensor voting to extract the depth surface. Experiments show that our algorithms are capable of producing comparable high quality depth maps which can be used for applications such as view interpolation.

[1] S. Baker, R. Szeliski, and P. Anandan, “A Layered Approach to Stereo Reconstruction,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 434-441, June 1998.
[2] R. Benosman and J. Devars, “Panoramic Stereovision Vensor,” Panoramic Vision: Sensors, Theory, and Applications, pp. 161-168, New York: Springer, 2001.
[3] S. Birchfield and C. Tomasi, “A Pixel Dissimilarity Measure that Is Insensitive to Image Sampling,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 401-406, Apr. 1998.
[4] R.C. Bolles, H.H. Baker, and D. Marimont, “Epipolar-Plane Image Analysis: An Approach to Determining Structure from Motion,” Int'l J. Computer Vision, vol. 1, pp. 7-55, 1987.
[5] R.T. Collins, “A Space-Sweep Approach to True Multi-Image Matching,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 358-363, June 1996.
[6] S.J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, “The Lumigraph,” Proc. ACM SIGGRAPH '96, pp. 43-54, 1996.
[7] R. Gupta and R. Hartley, “Linear Pushbroom Cameras,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 9, pp. 963-975, Sept. 1997.
[8] H. Ishiguro, M. Yamamoto, and S. Tsuji, “Omni-Directional Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 257-262, Feb. 1992.
[9] D. Jelinek and C.J. Taylor, “View Synthesis with Occlusion Reasoning Using Quasi-Sparse Feature Correspondences,” Proc. Seventh European Conf. Computer Vision, vol. 2, pp. 463-478, May 2002.
[10] T. Kanade, A. Yoshida, K. Oda, H. Kano, and M. Tanaka, “A Stereo Machine for Video-Rate Dense Depth Mapping and Its New Applications,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 196-202, June 1996.
[11] S.B. Kang and R. Szeliski, “3-D Scene Data Recovery Using Omnidirectional Multibaseline Stereo,” Int'l J. Computer Vision, vol. 25, no. 2, pp. 167-183, Nov. 1997.
[12] S.B. Kang, R. Szeliski, and J. Chai, “Handling Occlusions in Dense Multiview Stereo,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 103-110, Dec. 2001.
[13] R. Koch, M. Pollefeys, and L. Van Gool, “Multi Viewpoint Stereo from Uncalibrated Video Sequences,” Proc. Fifth European Conf. Computer Vision, vol. 1, pp. 55-71, June 1998.
[14] V. Kolmogorov and R. Zabih, “Multi-Camera Scene Reconstruction via Graph Cuts,” Proc. Seventh European Conf. Computer Vision, vol. 3, pp. 82-96, May 2002.
[15] K.N. Kutulakos and S.M. Seitz, “A Theory of Shape by Space Carving,” Proc. Seventh Int'l Conf. Computer Vision, pp. 307-314, Sept. 1999.
[16] M. Levoy and P. Hanrahan, “Light Field Rendering,” Proc. ACM SIGGRAPH '96, pp. 31-42, 1996.
[17] Y. Li, C.-K. Tang, and H.-Y. Shum, “Efficient Dense Depth Estimation from Dense Multiperspective Panoramas,” Proc. Eighth Int'l Conf. Computer Vision, pp. 119-126, 2001.
[18] L. McMillan and G. Bishop, “Plenoptic Modeling: An Image-Based Rendering System,” Proc. ACM SIGGRAPH '95, pp. 39-46, 1995.
[19] G. Medioni, M. Lee, and C. Tang, A Computational Framework for Feature Extraction and Segmentation. Amsterdam: Elsevier Science, 2000.
[20] M. Okutomi and T. Kanade, “A Multiple Baseline Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 4, pp. 353-363, Apr. 1993.
[21] T. Pajdla, “Stereo with Oblique Cameras,” Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 161-170, Apr. 2002.
[22] S. Peleg, M. Ben-Ezra, and Y. Pritch, “Omnistereo: Panoramic Stereo Imaging,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 279-290, Mar. 2001.
[23] S. Peleg and J. Herman, “Panoramic Mosaics by Manifold Projection,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 338-343, June 1997.
[24] P. Rademacher and G. Bishop, “Multiple-Center-of-Projection Images,” Proc. ACM SIGGRAPH '98, pp. 199-206, July 1998.
[25] D. Scharstein and R. Szeliski, “Stereo Matching with Nonlinear Diffusion,” Int'l J. Computer Vision, vol. 28, no. 2, pp. 155-174, July 1998.
[26] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int'l J. Computer Vision, vol. 47, no. 1, pp. 7-42, May 2002.
[27] S. Seitz and C. Dyer, “Photorealistic Scene Reconstruction by Voxel Coloring,” Int'l J. Computer Vision, vol. 25, no. 3, Nov. 1999.
[28] S. Seitz and J. Kim, “The Space of All Stereo Images,” Int'l J. Computer Vision, vol. 48, no. 1, pp. 21-38, June 2002.
[29] H. Shum, A. Kalai, and S. Seitz, “Omnivergent Stereo,” Proc. Int'l Conf. Computer Vision, pp. 22-29, 1999.
[30] H.-Y. Shum and L.-W. He, “Rendering with Concentric Mosaics,” Proc. ACM SIGGRAPH '99, pp. 299-306, 1999.
[31] H.-Y. Shum and R. Szeliski, “Stereo Reconstruction from Multiperspective Panoramas,” Proc. Seventh Int'l Conf. Computer Vision, pp. 14-21, 1999.
[32] R. Szeliski and P. Golland, “Stereo Matching with Transparency and Matting,” Int'l J. Computer Vision, special issue for Marr Prize papers, vol. 32, no. 1, pp. 45-61, Aug. 1999.
[33] R. Szeliski and D. Scharstein, “Symmetric Sub-Pixel Stereo Matching,” Proc. Seventh European Conf. Computer Vision, vol. 2, pp. 525-540, May 2002.
[34] D.N. Wood, et al.. “Multiperspective Panoramas for Cel Animation,” Proc. ACM SIGGRAPH '97, pp. 243-250, Aug. 1997.

Index Terms:
Multiperspective panorama, epipolar geometry, stereo, correspondence, tensor voting, plane sweep stereo, multibaseline stereo.
Citation:
Yin Li, Heung-Yeung Shum, Chi-Keung Tang, Richard Szeliski, "Stereo Reconstruction from Multiperspective Panoramas," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 45-62, Jan. 2004, doi:10.1109/TPAMI.2004.10002
Usage of this product signifies your acceptance of the Terms of Use.