CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2010 vol.32 Issue No.08 - August

Subscribe

Issue No.08 - August (2010 vol.32)

pp: 1392-1405

Victor Lempitsky , Microsoft Research, Cambridge

Carsten Rother , Micorsoft Research, Cambridge

Stefan Roth , Technische Universität Darmstadt, Darmstadt

Andrew Blake , Microsoft Research, Cambridge

ABSTRACT

The efficient application of graph cuts to Markov Random Fields (MRFs) with multiple discrete or continuous labels remains an open question. In this paper, we demonstrate one possible way of achieving this by using graph cuts to combine pairs of suboptimal labelings or solutions. We call this combination process the fusion move. By employing recently developed graph-cut-based algorithms (so-called QPBO-graph cut), the fusion move can efficiently combine two proposal labelings in a theoretically sound way, which is in practice often globally optimal. We demonstrate that fusion moves generalize many previous graph-cut approaches, which allows them to be used as building blocks within a broader variety of optimization schemes than were considered before. In particular, we propose new optimization schemes for computer vision MRFs with applications to image restoration, stereo, and optical flow, among others. Within these schemes the fusion moves are used 1) for the parallelization of MRF optimization into several threads, 2) for fast MRF optimization by combining cheap-to-compute solutions, and 3) for the optimization of highly nonconvex continuous-labeled MRFs with 2D labels. Our final example is a nonvision MRF concerned with cartographic label placement, where fusion moves can be used to improve the performance of a standard inference method (loopy belief propagation).

INDEX TERMS

Markov random fields, computer vision, combinatorial algorithms, graph algorithms, stereo, motion, image restoration.

CITATION

Victor Lempitsky, Carsten Rother, Stefan Roth, Andrew Blake, "Fusion Moves for Markov Random Field Optimization",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.32, no. 8, pp. 1392-1405, August 2010, doi:10.1109/TPAMI.2009.143REFERENCES

- [1] R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. Tappen, and C. Rother, "A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 6, pp. 1068-1080, June 2008.- [2] P.L. Hammer, "Some Network Flow Problems Solved with Pseudo Boolean Programming,"
Operations Research, vol. 13, pp. 388-399, 1965.- [3] D. Greig, B. Porteous, and A. Seheult, "Exact MAP Estimation for Binary Images,"
J. Royal Statistical Soc., Series B, vol. 51, no. 2, pp. 271-279, 1989.- [4] E. Boros and P.L. Hammer, "Pseudo Boolean Optimization,"
Discrete Applied Math., vol. 123, nos. 1-3, pp. 155-225, 2002.- [5] V. Kolmogorov and R. Zabih, "What Energy Functions Can Be Minimized via Graph Cuts?"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 147-159, Feb. 2004.- [6] J.S. Yedidia, W.T. Freeman, and Y. Weiss, "Understanding Belief Propagation and its Generalizations,"
Exploring Artificial Intelligence in the New Millennium, G. Lakemeyer and B. Nebel, eds., chapter 8, pp. 239-236, Morgan Kaufmann Publishers, 2003.- [7] H. Ishikawa, "Exact Optimization for Markov Random Fields with Convex Priors,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1333-1336, Oct. 2003.- [8] D. Schlesinger and B. Flach, "Transforming an Arbitrary Minsum Problem into a Binary One," Technical Report TUD-FI06-01, 2006.
- [9] P. Kohli, A. Shekhovtsov, C. Rother, V. Kolmogorov, and P. Torr, "On Partial Optimality in Multilabel MRFs,"
Proc. Int'l Conf. Machine Learning, 2008.- [10] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.- [11] O. Veksler, "Efficient Graph-Based Energy Minimization Methods in Computer Vision," PhD dissertation, Cornell Univ., 1999.
- [12] N. Komodakis, G. Tziritas, and N. Paragios, "Fast, Approximately Optimal Solutions for Single and Dynamic MRFs,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.- [13] O. Veksler, "Graph Cut Based Optimization for MRFs with Truncated Convex Priors,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.- [14] V. Kolmogorov and C. Rother, "Minimizing Non-Submodular Functions with Graph Cuts—A Review,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 7, pp. 1274-1279, July 2006.- [15] V. Lempitsky, C. Rother, and A. Blake, "LogCut—Efficient Graph Cut Optimization for Markov Random Fields,"
Proc. Int'l Conf. Computer Vision, 2007.- [16] V. Lempitsky, S. Roth, and C. Rother, "FusionFlow: Discrete-Continuous Optimization for Optical Flow Estimation,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.- [17] E. Boros, P.L. Hammer, and G. Tavares, "Preprocessing of Unconstrained Quadratic Binary Optimization," Technical Report RUTCOR RRR 10-2006, 2006.
- [18] C. Rother, V. Kolmogorov, V. Lempitsky, and M. Szummer, "Optimizing Binary MRFs via Extended Roof Duality,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.- [19] O.J. Woodford, P.H.S. Torr, I.D. Reid, and A.W. Fitzgibbon, "Global Stereo Reconstruction under Second Order Smoothness Priors,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.- [20] V. Kolmogorov and M.J. Wainwright, "On the Optimality of Tree-Reweighted Max-Product Message-Passing,"
Proc. Conf. Uncertainty in Artificial Intelligence, 2005.- [21] Y. Boykov, O. Veksler, and R. Zabih, "Markov Random Fields with Efficient Approximations,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 648-655, 1998.- [22] V. Kwatra, A. Schödl, I.A. Essa, G. Turk, and A.F. Bobick, "Graphcut Textures: Image and Video Synthesis Using Graph Cuts,"
ACM Trans. Graphics, vol. 22, no. 3, pp. 277-286, 2003.- [23] C. Rother, S. Kumar, V. Kolmogorov, and A. Blake, "Digital Tapestry,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 589-596, 2005.- [24] J.M. Winn and J. Shotton, "The Layout Consistent Random Field for Recognizing and Segmenting Partially Occluded Objects,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 37-44, 2006.- [25] O.J. Woodford, I.D. Reid, P.H.S. Torr, and A.W. Fitzgibbon, "On New View Synthesis Using Multiview Stereo,"
Proc. British Machine Vision Conf., 2007.- [26] A. Billionnet and M. Minoux, "Maximizing a Supermodular Pseudo Boolean Function: A Polynomial Algorithm for Supermodular Cubic Functions,"
Discrete Applied Math., vol. 12, no. 1, pp. 1-11, 1985.- [27] D. Scharstein and C. Pal, "Learning Conditional Random Fields for Stereo,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.- [28] P.F. Felzenszwalb and D.P. Huttenlocher, "Efficient Belief Propagation for Early Vision,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 261-268, 2004.- [29] V. Kolmogorov, "Convergent Tree-Reweighted Message Passing for Energy Minimization,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1568-1583, Oct. 2006.- [30] S. Baker, D. Scharstein, J. Lewis, S. Roth, M. Black, and R. Szeliski, "A Database and Evaluation Methodology for Optical Flow,"
Proc. Int'l Conf. Computer Vision, 2007.- [31] S. Roth and M.J. Black, "On the Spatial Statistics of Optical Flow,"
Int'l J. Computer Vision, vol. 74, no. 1, pp. 33-50, 2007.- [32] N. Papenberg, A. Bruhn, T. Brox, S. Didas, and J. Weickert, "Highly Accurate Optic Flow Computation with Theoretically Justified Warping,"
Int'l J. Computer Vision, vol. 67, no. 2, pp. 141-158, Apr. 2006.- [33] W. Trobin, T. Pock, D. Cremers, and H. Bischof, "Continuous Energy Minimization via Repeated Binary Fusion,"
Proc. European Conf. Computer Vision, 2008.- [34] B.D. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision,"
Proc. Int'l Joint Conf. Artificial Intelligence, pp. 674-679, 1981.- [35] B.K.P. Horn and B.G. Schunck, "Determining Optical Flow,"
Artificial Intelligence, vol. 17, nos. 1-3, pp. 185-203, Aug. 1981.- [36] S. Birchfield, B. Natarjan, and C. Tomasi, "Correspondence as Energy-Based Segmentation,"
Image and Vision Computing, vol. 25, no. 8, pp. 1329-1340, 2007.- [37] J. Besag, "On the Statistical Analysis of Dirty Pictures,"
J. Royal Statistical Soc., vol. B-48, no. 3, pp. 259-302, 1986.- [38] C.E. Rasmussen, "minimize.m," http://www.kyb.tuebingen. mpg.de/bs/ people/carl/codeminimize/, Sept. 2006.
- [39] A. Wolff, "The Map-Labeling Bibliography," http://i11www.iti. uni-karlsruhe.de/map-labeling bibliography/, 2009.
- [40] J. Christensen, J. Marks, and S. Shieber, "An Empirical Study of Algorithms for Point-Feature Label Placement,"
ACM Trans. Graphics, vol. 14, no. 3, pp. 203-232, 1995.- [41] H.Y. Jung, K.M. Lee, and S.U. Lee, "Toward Global Minimum through Combined Local Minima,"
Proc. European Conf. Computer Vision, 2008. |