The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - February (2008 vol.30)
pp: 328-341
ABSTRACT
This paper describes the Semi-Global Matching (SGM) stereo method. It uses a pixelwise, Mutual Information based matching cost for compensating radiometric differences of input images. Pixelwise matching is supported by a smoothness constraint that is usually expressed as a global cost function. SGM performs a fast approximation by pathwise optimizations from all directions. The discussion also addresses occlusion detection, subpixel refinement and multi-baseline matching. Additionally, postprocessing steps for removing outliers, recovering from specific problems of structured environments and the interpolation of gaps are presented. Finally, strategies for processing almost arbitrarily large images and fusion of disparity images using orthographic projection are proposed.A comparison on standard stereo images shows that SGM is among the currently top-ranked algorithms and is best, if subpixel accuracy is considered. The complexity is linear to the number of pixels and disparity range, which results in a runtime of just 1-2s on typical test images. An in depth evaluation of the Mutual Information based matching cost demonstrates a tolerance against a wide range of radiometric transformations. Finally, examples of reconstructions from huge aerial frame and pushbroom images demonstrate that the presented ideas are working well on practical problems.
INDEX TERMS
stereo, mutual information, global optimization, multi-baseline
CITATION
Heiko Hirschm?, "Stereo Processing by Semiglobal Matching and Mutual Information", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 2, pp. 328-341, February 2008, doi:10.1109/TPAMI.2007.1166
REFERENCES
[1] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int'l J.Computer Vision, vol. 47, nos. 1/2/3, pp. 7-42, Apr.-June 2002.
[2] S. Birchfield and C. Tomasi, “Depth Discontinuities by Pixel-to-Pixel Stereo,” Proc. Sixth IEEE Int'l Conf. Computer Vision, pp. 1073-1080, Jan. 1998.
[3] A. Klaus, M. Sormann, and K. Karner, “Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure,” Proc. Int'l Conf. Pattern Recognition, 2006.
[4] P. Viola and W.M. Wells, “Alignment by Maximization of Mutual Information,” Int'l J. Computer Vision, vol. 24, no. 2, pp. 137-154, 1997.
[5] G. Egnal, “Mutual Information as a Stereo Correspondence Measure,” Technical Report MS-CIS-00-20, Computer and Information Science, Univ. of Pennsylvania, 2000.
[6] J. Kim, V. Kolmogorov, and R. Zabih, “Visual Correspondence Using Energy Minimization and Mutual Information,” Proc. Int'l Conf. Computer Vision, Oct. 2003.
[7] C.L. Zitnick, S.B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, “High-Quality Video View Interpolation Using a Layered Representation,” Proc. ACM SIGGRAPH '04, 2004.
[8] H. Hirschmüller, P.R. Innocent, and J.M. Garibaldi, “Real-Time Correlation-Based Stereo Vision with Reduced Border Errors,” Int'l J. Computer Vision, vol. 47, nos. 1/2/3, pp. 229-246, Apr.-June 2002.
[9] M. Bleyer and M. Gelautz, “A Layered Stereo Matching Algorithm Using Image Segmentation and Global Visibility Constraints,” ISPRS J. Photogrammetry and Remote Sensing, vol. 59, no. 3, pp. 128-150, 2005.
[10] K.-J. Yoon and I.-S. Kweon, “Adaptive Support-Weight Approach for Correspondence Search,” IEEE Trans. Pattern Matching and Machine Intelligence, vol. 28, no. 4, pp. 650-656, Apr. 2006.
[11] Q. Yang, L. Wang, R. Yang, H. Stewenius, and D. Nister, “Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation and Occlusion Handling,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2006.
[12] C. Lei, J. Selzer, and Y.-H. Yang, “Region-Tree Based Stereo Using Dynamic Programming Optimization,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2006.
[13] V. Kolmogorov and R. Zabih, “Computing Visual Correspondence with Occlusions Using Graph Cuts,” Proc. Int'l Conf. Computer Vision, vol. 2, pp. 508-515, 2001.
[14] J. Sun, Y. Li, S. Kang, and H.-Y. Shum, “Symmetric Stereo Matching for Occlusion Handling,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 399-406, June 2005.
[15] G. Van Meerbergen, M. Vergauwen, M. Pollefeys, and L. Van Gool, “A Hierarchical Symmetric Stereo Algorithm Using Dynamic Programming,” Int'l J. Computer Vision, vol. 47, nos. 1/2/3, pp. 275-285, Apr.-June 2002.
[16] O. Veksler, “Stereo Correspondence by Dynamic Programming on a Tree,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 384-390, June 2005.
[17] H. Hirschmüller, “Stereo Vision Based Mapping and Immediate Virtual Walkthroughs,” PhD dissertation, School of Computing, De Montfort Univ., June 2003.
[18] D. Scharstein and R. Szeliski, “High-Accuracy Stereo Depth Maps Using Structured Light,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 195-202, June 2003.
[19] H. Hirschmüller, “Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 807-814, June 2005.
[20] H. Hirschmüller, “Stereo Vision in Structured Environments by Consistent Semi-Global Matching,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2006.
[21] R. Gupta and R.I. Hartley, “Linear Pushbroom Cameras,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 9, pp.963-975, Sept. 1997.
[22] H. Hirschmüller, F. Scholten, and G. Hirzinger, “Stereo Vision Based Reconstruction of Huge Urban Areas from an Airborne Pushbroom Camera (HRSC),” Proc. 27th Symp. German Assoc. for Pattern Recognition, pp. 58-66, Aug./Sept. 2005.
[23] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, 2001.
[24] E.R. Davis, Machine Vision: Theory, Algorithms, Practicalities, second ed. Academic Press, 1997.
[25] D. Comaniciu and P. Meer, “Mean Shift: A Robust Approach Toward Feature Space Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 1-18, May 2002.
[26] C. Strecha, R. Fransens, and L. Van Gool, “Combined Depth and Outlier Estimation in Multi-View Stereo,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2006.
[27] F. Wewel, F. Scholten, and K. Gwinner, “High Resolution Stereo Camera (HRSC)—Multispectral 3D-Data Acquisition and Photogrammetric Data Processing,” Canadian J. Remote Sensing, vol. 26, no. 5, pp. 466-474, 2000.
45 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool