This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors
June 2008 (vol. 30 no. 6)
pp. 1068-1080
ASCII Text
x 
 
Richard Szeliski, Ramin Zabih, Daniel Scharstein, Olga Veksler, Vladimir Kolmogorov, Aseem Agarwala, Marshall Tappen, Carsten Rother, "A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 6, pp. 1068-1080, June, 2008.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2007.70844,
author = {Richard Szeliski and Ramin Zabih and Daniel Scharstein and Olga Veksler and Vladimir Kolmogorov and Aseem Agarwala and Marshall Tappen and Carsten Rother},
title = {A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {30},
number = {6},
issn = {0162-8828},
year = {2008},
pages = {1068-1080},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2007.70844},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors
IS - 6
SN - 0162-8828
SP1068
EP1080
EPD - 1068-1080
A1 - Richard Szeliski,
A1 - Ramin Zabih,
A1 - Daniel Scharstein,
A1 - Olga Veksler,
A1 - Vladimir Kolmogorov,
A1 - Aseem Agarwala,
A1 - Marshall Tappen,
A1 - Carsten Rother,
PY - 2008
KW - Performance evaluation of algorithms and systems
KW - Markov random fields
KW - Global optimization
KW - Graph cuts
KW - Belief propagation
VL - 30
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Among the most exciting advances in early vision has been the development of efficient energy minimization algorithms for pixel-labeling tasks such as depth or texture computation. It has been known for decades that such problems can be elegantly expressed as Markov random fields, yet the resulting energy minimization problems have been widely viewed as intractable. Recently, algorithms such as graph cuts and loopy belief propagation (LBP) have proven to be very powerful: for example, such methods form the basis for almost all the top-performing stereo methods. However, the tradeoffs among different energy minimization algorithms are still not well understood. In this paper we describe a set of energy minimization benchmarks and use them to compare the solution quality and running time of several common energy minimization algorithms. We investigate three promising recent methods?graph cuts, LBP, and tree-reweighted message passing?in addition to the well-known older iterated conditional modes (ICM) algorithm. Our benchmark problems are drawn from published energy functions used for stereo, image stitching, interactive segmentation, and denoising. We also provide a general-purpose software interface that allows vision researchers to easily switch between optimization methods. Benchmarks, code, images, and results are available at http://vision.middlebury.edu/MRF/.

[1] A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen, “Interactive Digital Photomontage,” ACM Trans. Graphics, vol. 23, no. 3, pp. 294-302, 2004.
[2] A. Agarwala, C. Zheng, C. Pal, M. Agrawala, M. Cohen, B. Curless, D. Salesin, and R. Szeliski, “Panoramic Video Textures,” ACM Trans. Graphics, vol. 24, no. 3, pp. 821-827, 2005.
[3] R. Ahuja, Ö. Ergun, J. Orlin, and A. Punnen, “A Survey of Very Large-Scale Neighborhood Search Techniques,” Discrete Applied Math., vol. 123, nos. 1-3, pp. 75-102, 2002.
[4] S. Barnard, “Stochastic Stereo Matching over Scale,” Int'l J. Computer Vision, vol. 3, no. 1, pp. 17-32, 1989.
[5] J. Besag, “On the Statistical Analysis of Dirty Pictures (with discussion),” J. Royal Statistical Soc., Series B, vol. 48, no. 3, pp. 259-302, 1986.
[6] S. Birchfield and C. Tomasi, “A Pixel Dissimilarity Measure That Is Insensitive to Image Sampling,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 401-406, Apr. 1998.
[7] A. Blake and A. Zisserman, Visual Reconstruction. MIT Press, 1987.
[8] Y. Boykov and M.-P. Jolly, “Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images,” Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 1, pp. 105-112, 2001.
[9] Y. Boykov and V. Kolmogorov, “An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1124-1137, Sept. 2004.
[10] Y. Boykov, O. Veksler, and R. Zabih, “Markov Random Fields with Efficient Approximations,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 648-655, 1998.
[11] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[12] C. Buehler, S. Gortler, M. Cohen, and L. McMillan, “Minimal Surfaces for Stereo Vision,” Proc. Seventh European Conf. Computer Vision, vol. 3, pp. 885-899, 2002.
[13] P. Chou and C. Brown, “The Theory and Practice of Bayesian Image Labeling,” Int'l J. Computer Vision, vol. 4, no. 3, pp. 185-210, 1990.
[14] K. Driesen and U. Hölzle, “The Direct Cost of Virtual Function Calls in C++,” Proc. ACM SIGPLAN Conf. Object-Oriented Programming Systems, Languages, and Applications, pp. 306-323, 1996.
[15] A. Efros and W. Freeman, “Image Quilting for Texture Synthesis and Transfer,” Proc. ACM SIGGRAPH '01, pp. 341-346, 2001.
[16] P. Felzenszwalb and D. Huttenlocher, “Efficient Belief Propagation for Early Vision,” Int'l J. Computer Vision, vol. 70, no. 1, pp. 41-54, 2006.
[17] W. Freeman, E. Pasztor, and O. Carmichael, “Learning Low-Level Vision,” Int'l J. Computer Vision, vol. 40, no. 1, pp. 25-47, 2000.
[18] B. Frey and D. MacKay, “A Revolution: Belief Propagation in Graphs with Cycles,” Advances in Neural Information Processing Systems, 1997.
[19] S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, pp. 721-741, 1984.
[20] S. Geman and C. Graffigne, “Markov Random Field Image Models and Their Applications to Computer Vision,” Proc. Int'l Congress of Math., pp. 1496-1517, 1986.
[21] D. Greig, B. Porteous, and A. Seheult, “Exact Maximum A Posteriori Estimation for Binary Images,” J. Royal Statistical Soc., Series B, vol. 51, no. 2, pp. 271-279, 1989.
[22] P. Hammer, “Some Network Flow Problems Solved with Pseudo-Boolean Programming,” Operations Research, vol. 13, pp. 388-399, 1965.
[23] D. Hochbaum, “An Efficient Algorithm for Image Segmentation, Markov Random Fields and Related Problems,” J. ACM, vol. 48, no. 4, pp. 686-701, 2001.
[24] H. Ishikawa, “Exact Optimization for Markov Random Fields with Convex Priors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1333-1336, Oct. 2003.
[25] P. Kohli and P. Torr, “Dynamic Graph Cuts for Efficient Inference in Markov Random Fields,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2079-2088, Dec. 2007.
[26] V. Kolmogorov, “Convergent Tree-Reweighted Message Passing for Energy Minimization,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1568-1583, Oct. 2006.
[27] V. Kolmogorov, A. Criminisi, A. Blake, G. Cross, and C. Rother, “Probabilistic Fusion of Stereo with Color and Contrast for Bilayer Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1480-1492, Sept. 2006.
[28] V. Kolmogorov and C. Rother, “Comparison of Energy Minimization Algorithms for Highly Connected Graphs,” Proc. Ninth European Conf. Computer Vision, pp. 1-15, 2006.
[29] V. Kolmogorov and C. Rother, “Minimizing Nonsubmodular Functions with Graph Cuts—A Review,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 7, pp. 1274-1279, 2007.
[30] V. Kolmogorov and R. Zabih, “Multi-Camera Scene Reconstruction via Graph Cuts,” Proc. Seventh European Conf. Computer Vision, pp. 82-96, 2002.
[31] V. Kolmogorov and R. Zabih, “What Energy Functions can be Minimized via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 147-159, Feb. 2004.
[32] N. Komodakis and G. Tziritas, “Approximate Labeling via Graph Cuts Based on Linear Programming,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 8, pp. 1436-1453, Aug. 2007.
[33] N. Komodakis, G. Tziritas, and N. Paragios, “Fast, Approximately Optimal Solutions for Single and Dynamic MRFs,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 2007.
[34] V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, “Graphcut Textures: Image and Video Synthesis Using Graph Cuts,” ACM Trans. Graphics, pp. 277-286, 2003.
[35] S. Li, Markov Random Field Modeling in Computer Vision. Springer, 1995.
[36] M. Lin and C. Tomasi, “Surfaces with Occlusions from Layered Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp. 1073-1078, Aug. 2004.
[37] T. Meltzer, C. Yanover, and Y. Weiss, “Globally Optimal Solutions for Energy Minimization in Stereo Vision Using Reweighted Belief Propagation,” Proc. 11th IEEE Int'l Conf. Computer Vision, pp. 428-435, 2005.
[38] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
[39] C. Rother, V. Kolmogorov, and A. Blake, “‘GrabCut’—Interactive Foreground Extraction Using Iterated Graph Cuts,” ACM Trans. Graphics, vol. 23, no. 3, pp. 309-314, 2004.
[40] C. Rother, V. Kolmogorov, V. Lempitsky, and M. Szummer, “Optimizing Binary MRFs via Extended Roof Duality,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 2007.
[41] C. Rother, S. Kumar, V. Kolmogorov, and A. Blake, “Digital Tapestry,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 589-596, 2005.
[42] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 7-42, 2002.
[43] D. Snow, P. Viola, and R. Zabih, “Exact Voxel Occupancy with Graph Cuts,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 345-352, 2000.
[44] R. Szeliski, “Locally Adapted Hierarchical Basis Preconditioning,” ACM Trans. Graphics, vol. 25, no. 3, pp. 1135-1143, 2006.
[45] M. Tappen and W. Freeman, “Comparison of Graph Cuts with Belief Propagation for Stereo, Using Identical MRF Parameters,” Proc. Ninth IEEE Int'l Conf. Computer Vision, pp. 900-907, 2003.
[46] O. Veksler, “Graph Cut Based Optimization for MRFs with Truncated Convex Priors,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2007.
[47] M. Wainwright, T. Jaakkola, and A. Willsky, “Tree Consistency and Bounds on the Performance of the Max-Product Algorithm and Its Generalizations,” Statistics and Computing, vol. 14, pp. 143-166, Apr. 2004.
[48] M. Wainwright, T. Jaakkola, and A. Willsky, “Map Estimation via Agreement on Trees: Message-Passing and Linear Programming,” IEEE Trans. Information Theory, vol. 51, no. 11, pp. 3697-3717, 2005.
[49] Y. Weiss and W. Freeman, “On the Optimality of Solutions of the Max-Product Belief Propagation Algorithm in Arbitrary Graphs,” IEEE Trans. Information Theory, vol. 47, no. 2, pp. 723-735, 2001.
[50] J. Yedidia, W. Freeman, and Y. Weiss, “Generalized Belief Propagation,” Advances in Neural Information Processing Systems, pp. 689-695, 2000.

Index Terms:
Performance evaluation of algorithms and systems, Markov random fields, Global optimization, Graph cuts, Belief propagation
Citation:
Richard Szeliski, Ramin Zabih, Daniel Scharstein, Olga Veksler, Vladimir Kolmogorov, Aseem Agarwala, Marshall Tappen, Carsten Rother, "A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 6, pp. 1068-1080, June 2008, doi:10.1109/TPAMI.2007.70844
Usage of this product signifies your acceptance of the Terms of Use.