The Community for Technology Leaders
Subscribe
Issue No.11 - November (2010 vol.32)
pp: 2071-2084
Rami Ben-Ari , Orbotech Ltd., Yavneh
Nir Sochen , Tel-Aviv University, Ramat-Aviv
ABSTRACT
This paper addresses the problem of correspondence establishment in binocular stereo vision. We suggest a novel spatially continuous approach for stereo matching based on the variational framework. The proposed method suggests a unique regularization term based on Mumford-Shah functional for discontinuity preserving, combined with a new energy functional for occlusion handling. The evaluation process is based on concurrent minimization of two coupled energy functionals, one for domain segmentation (occluded versus visible) and the other for disparity evaluation. In addition to a dense disparity map, our method also provides an estimation for the half-occlusion domain and a discontinuity function allocating the disparity/depth boundaries. Two new constraints are introduced improving the revealed discontinuity map. The experimental tests include a wide range of real data sets from the Middlebury stereo database. The results demonstrate the capability of our method in calculating an accurate disparity function with sharp discontinuities and occlusion map recovery. Significant improvements are shown compared to a recently published variational stereo approach. A comparison on the Middlebury stereo benchmark with subpixel accuracies shows that our method is currently among the top-ranked stereo matching algorithms.
INDEX TERMS
Stereo matching, Mumford-Shah functional, variational stereo vision, occlusion handling, Total Variation.
CITATION
Rami Ben-Ari, Nir Sochen, "Stereo Matching with Mumford-Shah Regularization and Occlusion Handling", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 11, pp. 2071-2084, November 2010, doi:10.1109/TPAMI.2010.32
REFERENCES
 [1] R. Alicandro, A. Braides, and J. Shah, "Free-Discontinuity Problems via Functionals Involving $l^1$ -Norm of the Gradient and Their Approximation," Interface and Free Boundaries, vol. 1, pp. 17-37, 1999. [2] L. Alvarez, R. Deriche, J. Sánchez, and J. Weickert, "Dense Disparity Map Estimation Respecting Image Discontinuities: A PDE and Scale-Space Based Approach," J. Visual Comm. and Image Representation, vol. 13, pp. 3-21, 2002. [3] L. Ambrosio and V.M. Tortorelli, "Approximation of Functionals Depending on Jumps by Elliptic Functionals via $\Gamma$ -Convergence," Comm. Pure and Applied Math., vol. 43, pp. 999-1036, 1990. [4] T. Amiaz and N. Kiryati, "Dense Discontinuous Optical Flow via Contour Based Segmentation," Proc. IEEE Int'l Conf. Image Processing, vol. 3, pp. 1264-1267, 2005. [5] A nonymous, "Stereo Matching Based on Under and Over-Segmentation with Occlusion Handling," http://vision.middle bury.edu/stereoeval /, 2010. [6] L. Bar, A. Brook, N. Sochen, and N. Kiryati, "Deblurring of Color Images Corrupted by Impulsive Noise," IEEE Trans. Image Processing, vol. 16, no. 4, pp. 1101-1111, Apr. 2007. [7] R. Ben-Ari and N. Sochen, "Variational Stereo Vision with Sharp Discontinuities and Occlusion Handling," Proc. IEEE Int'l Conf. Computer Vision, pp. 1-7, 2007. [8] M. Bleyer and M. Gelautz, "A Layered Stereo Algorithm Using Image Segmentation and Global Visibility Constraints," Proc. IEEE Int'l Conf. Image Processing, vol. 5, pp. 2997-3000, 2004. [9] M.Z. Brown, D. Burschka, and G.D. Hager, "Advances in Computational Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 993-1008, Aug. 2003. [10] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, "High Accuracy Optical Flow Estimation Based on a Theory for Warping," Proc. Eighth European Conf. Computer Vision, pp. 25-36, 2004. [11] T. Chan and L. Vese, "Active Contours without Edges," IEEE Trans. Image Processing, vol. 10, no. 2, pp. 266-277, Feb. 2001. [12] G. Dal Maso, "An Introduction to $\Gamma$ -Convergence," Progress in Nonlinear Differential Equations and Their Applications, Birkhauser, 1993. [13] U.R. Dhond and J.K. Aggarwal, "Stereo Matching in Presence of Narrow Occluding Objects Using Dynamic Disparity Search," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 7, pp. 719-724, July 1995. [14] L.C. Evans and R.F. Gariepy, Measure Theory and Fine Properties of Functions. CRC Press, 1992. [15] L. Di Stefano, F. Tombari, S. Mattoccia, "Segmentation-Based Adaptive Support for Accurate Stereo Correspondence," Proc. IEEE Pacific-Rim Symp. Image and Video Technology, 2007. [16] A. Fusiello, V. Roberto, and E. Trucco, "Efficient Stereo with Multiple Windowing," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 858-863, 1997. [17] S. Gehrig and U. Franke, "Improving Sub-Pixel Accuracy for Long Range Stereo," Proc. IEEE Int'l Conf. Computer Vision VRML Workshop, 2007. [18] D. Geiger, B. Landendorf, and A.L. Yuille, "Occlusions and Binocular Stereo," Int'l J. Computer Vision, vol. 14, no. 3, pp. 211-226, 1995. [19] H. Hirschmüller, "Stereo Processing by Semi-Global Matching and Mutual Information," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328-341, Feb. 2008. [20] T. Kanade and M. Okutomi, "A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiments," IEEE Trans. Pattern Recognition and Machine Intelligence, vol. 16, no. 9, pp. 920-932, Sept. 1994. [21] H. Kim and K. Sohn, "Hierarchical Disparity Estimation with Energy Based Regularization," Proc. 10th IEEE Int'l Conf. Image Processing, vol. 1, pp. 373-376, 2003. [22] A. Klaus, M. Sormann, and K. Karner, "Segment-Based Stereo Matching Using Belief Propagation and Self-Adapting Dissimilarity Measure," Proc. Int'l Conf. Pattern Recognition, vol. 3, pp. 15-18, 2006. [23] M. Klodt, T. Schoenemann, K. Kolev, M. Schikora, and D. Cremers, "An Experimental Comparison of Discrete and Continuous Shape Optimization Methods," Proc. European Conf. Computer Vision, 2008. [24] K. Kolmogorov and R. Zabih, "Computing Visual Correspondence with Occlusions Using Graph Cuts," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 508-515, 2001. [25] A. Kumar, C.V.S. Haker, S. Zucker, and A. Tannenbaum, "Stereo Disparity and $l^1$ Minimization," Proc. 36th IEEE Conf. Decision and Control, vol. 2, pp. 1125-1129, 1997. [26] C. Lei, J. Selzer, and Y.H. Yang, "Region Tree Based Stereo Using Dynamic Programming Optimization," Proc. IEEE Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2378-2385, 2006. [27] A.R. Mansouri, A. Mitiche, and J. Konard, "Selective Image Diffusion: Application to Disparity Estimation," Proc. IEEE Int'l Conf. Image Processing, vol. 3, pp. 114-118, 1998. [28] D. Marr and T.A. Poggio, "Cooperative Computation of Stereo Disparity," Science, vol. 194, no. 4262, pp. 283-287, 1976. [29] S. Mattoccia, S. Giardino, and A. Gambini, "Accurate and Efficient Cost Aggregation Strategy for Stereo Correspondence Based on Approximated Joint Bilateral Filtering," Proc. Asian Conf. Computer Vision, 2009. [30] W. Miled and J.C. Pesquet, "Disparity Map Estimation Using a Total Variation Bound," Proc. Third Canadian Conf. Computer and Robot Vision. [31] D. Mumford and J. Shah, "Optimal Approximations by Piecewise Smooth Functions and Associated Variational Problems," Comm. Pure and Applied Math., vol. 42, pp. 577-684, 1989. [32] S. Osher and J.A. Sethian, "Fronts Propagating with Curvature-Dependent Speed: Algorithms Based on Hamilton-Jacobi Formulations," J. Computational Physics, vol. 79, pp. 577-684, 1988. [33] Middlebury Optical Flow Benchmark Page. http://vision. middlebury.eduflow/, 2010. [34] Middlebury Stereo Benchmark Page. http://vision.middlebury. edustereo/, 2010. [35] T. Pock, T. Schoenemann, G. Graber, H. Bischof, and D. Cremers, "A Convex Formulation of Continuous Multi-Label Problems," Proc. European Conf. Computer Vision, 2008. [36] L. Robert and R. Deriche, "Dense Depth Map Reconstruction: A Minimization and Regularization Approach Which Preserves Discontinuities," Proc. European Conf. Computer Vision, pp. 439-451, 1996. [37] D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two Frame Stereo Correspondence Algorithms," Int'l J. Computer Vision, vol. 47, no. 1, pp. 7-42, 2002. [38] C. Schmid and A. Zisserman, "The Geometry and Matching of Curves in Multiple Views," Proc. European Conf. Computer Vision, pp. 104-118, 1998. [39] J. Shah, "Segmentation By Nonlinear Diffusion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 202-207, 1991. [40] J. Shah, "A Nonlinear Diffusion Model for Discontinuous Disparity and Half-Occlusions in Stereo," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 34-40, 1993. [41] N. Slesareva, A. Bruhn, and J. Weickert, "Optical Flow Goes Stereo: A Variational Method for Estimating Discontinuity-Preserving Dense Disparity Maps," Proc. 27th DAGM Symp. Pattern Recognition, W. Kropatsch, R. Sablatnig, and A. Hanbury, eds., pp. 33-40, 2005. [42] J. Sun, Y. Li, S.B. Kang, and H.-Y. Shum, "Symmetric Stereo Matching for Occlusion Handling," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 399-406, 2005. [43] R. Kimmel, T. Nir, and A.M. Bruckstein, "Over-Parametrized Variational Optical Flow," Int'l J. Computer Vision, vol. 76, no. 2, pp. 205-216, 2008. [44] A.N. Tikhonov and V.Y. Arsenin, Solutions of Ill-Posed Problems. Winston and Sons, 1977. [45] V. Venkateswar and R. Chellapa, "Hierarchical Stereo and Motion Correspondence Using Feature Grouping," Int'l J. Computer Vision, vol. 15, pp. 245-269, 1995. [46] L.A. Vese and T.F. Chan, "A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model," Int'l J. Computer Vision, vol. 50, no. 3, pp. 271-293, 2002. [47] J. Weickert, Anisotropic Diffusion in Image Processing. Teubner Stuttgart, 1998. [48] Q. Yáng, R.Y.L. Wang, H. Stewénius, and D. Nistér, "Stereo Matching with Color-Weighted Correlation, Hierarchial Belief Propagation and Occlusion Handling," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2347-2354, 2006. [49] Q. Yáng, R. Yang, J. Davis, and D. Nistér, "Spatial-Depth Super Resolution for Range Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007. [50] K. Yoon and I. Kweon, "Locally Adaptive Support-Weight Approach for Visual Correspondence Search," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 924-931, 2005.