This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Revisiting the Linear Programming Relaxation Approach to Gibbs Energy Minimization and Weighted Constraint Satisfaction
August 2010 (vol. 32 no. 8)
pp. 1474-1488
ASCII Text
x 
 
Tomáš Werner, "Revisiting the Linear Programming Relaxation Approach to Gibbs Energy Minimization and Weighted Constraint Satisfaction," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 8, pp. 1474-1488, August, 2010.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2009.134,
author = {Tomáš Werner},
title = {Revisiting the Linear Programming Relaxation Approach to Gibbs Energy Minimization and Weighted Constraint Satisfaction},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {32},
number = {8},
issn = {0162-8828},
year = {2010},
pages = {1474-1488},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2009.134},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Revisiting the Linear Programming Relaxation Approach to Gibbs Energy Minimization and Weighted Constraint Satisfaction
IS - 8
SN - 0162-8828
SP1474
EP1488
EPD - 1474-1488
A1 - Tomáš Werner,
PY - 2010
KW - Weighted constraint satisfaction
KW - Gibbs distribution
KW - graphical model
KW - Markov random field
KW - linear programming relaxation
KW - marginal polytope
KW - cut polytope
KW - cutting-plane algorithm
KW - global constraint
KW - supermodularity
KW - tree-reweighted max-product.
VL - 32
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Tomáš Werner, Czech Technical University, Praha
We present a number of contributions to the LP relaxation approach to weighted constraint satisfaction (= Gibbs energy minimization). We link this approach to many works from constraint programming, which relation has so far been ignored in machine vision and learning. While the approach has been mostly considered only for binary constraints, we generalize it to n-ary constraints in a simple and natural way. This includes a simple algorithm to minimize the LP-based upper bound, n-ary max-sum diffusion—however, we consider using other bound-optimizing algorithms as well. The diffusion iteration is tractable for a certain class of high-arity constraints represented as a black box, which is analogical to propagators for global constraints CSP. Diffusion exactly solves permuted n-ary supermodular problems. A hierarchy of gradually tighter LP relaxations is obtained simply by adding various zero constraints and coupling them in various ways to existing constraints. Zero constraints can be added incrementally, which leads to a cutting-plane algorithm. The separation problem is formulated as finding an unsatisfiable subproblem of a CSP.

[1] P. Meseguer, F. Rossi, and T. Schiex, "Soft Constraints," Handbook of Constraint Programming, chapter 9, Elsevier, 2006.
[2] P.L. Hammer, P. Hansen, and B. Simeone, "Roof Duality, Complementation and Persistency in Quadratic 0-1 Optimization," Math. Programming, vol. 28, pp. 121-155, 1984.
[3] M.I. Schlesinger, "Syntactic Analysis of Two-Dimensional Visual Signals in Noisy Conditions," Cybernetics and Systems Analysis, vol. 12, no. 4, pp. 612-628, 1976, translation from Russian, Sintaksicheskiy analiz dvumernykh zritelnikh signalov v usloviyakh pomekh, Kibernetika, vol. 12, no. 4, pp. 113-130, 1976.
[4] V.A. Kovalevsky and V.K. Koval, "A Diffusion Algorithm for Decreasing Energy of Max-Sum Labeling Problem," approx. 1975, Glushkov Inst. of Cybernetics, unpublished.
[5] M.I. Schlesinger, "personal communication, 2000-2005.
[6] V.K. Koval and M.I. Schlesinger, "Dvumernoe Programmirovanie v Zadachakh Analiza Izobrazheniy (Two-Dimensional Programming in Image Analysis Problems)," USSR Academy of Science, Automatics, and Telemechanics, vol. 8, pp. 149-168, 1976 (in Russian).
[7] T. Werner, "A Linear Programming Approach to Max-Sum Problem: A Review," Technical Report CTU-CMP-2005-25, Center for Machine Perception, Czech Technical Univ., Dec. 2005.
[8] T. Werner, "A Linear Programming Approach to Max-Sum Problem: A Review," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 7, pp. 1165-1179, July 2007.
[9] T. Werner, "High-Arity Interactions, Polyhedral Relaxations, and Cutting Plane Algorithm for Soft Constraint Optimisation (MAP-MRF)," Proc. IEEE Computer Vision and Pattern Recognition Conf., June 2008.
[10] F. Rossi, P. van Beek, and T. Walsh, Handbook of Constraint Programming. Elsevier, 2006.
[11] D. Cohen, M. Cooper, P. Jeavons, and A. Krokhin, "The Complexity of Soft Constraint Satisfaction," Artificial Intelligence, vol. 170, pp. 983-1016, 2006.
[12] M.C. Cooper, S. de Givry, M. Sánchez, T. Schiex, and M. Zytnicki, "Virtual Arc Consistency for Weighted CSP," Proc. Conf. Artificial Intelligence, pp. 253-258, July 2008.
[13] U. Montanari, "Networks of Constraints: Fundamental Properties and Application to Picture Processing," Information Science, vol. 7, pp. 95-132, 1974.
[14] A. Rosenfeld, R.A. Hummel, and S.W. Zucker, "Scene Labeling by Relaxation Operations," IEEE Trans. Systems, Man, and Cybernetics, vol. 6, no. 6, pp. 420-433, June 1976.
[15] D. Schlesinger, "Exact Solution of Permuted Submodular MinSum Problems," Proc. Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 28-38, 2007.
[16] M.J. Wainwright and M.I. Jordan, "Graphical Models, Exponential Families, and Variational Inference," Technical Report 649, Dept. of Statistics, Univ. of California, Berkeley, 2003.
[17] M. Wainwright, T. Jaakkola, and A. Willsky, "MAP Estimation via Agreement on (Hyper)Trees: Message Passing and Linear Programming Approaches," IEEE Trans. Information Theory, vol. 51, no. 11, pp. 3697-3717, Nov. 2005.
[18] M.J. Wainwright and M.I. Jordan, "Graphical Models, Exponential Families, and Variational Inference," Foundations and Trends in Machine Learning, vol. 1, nos. 1/2, pp. 1-305, 2008.
[19] V. Kolmogorov, "Convergent Tree-Reweighted Message Passing for Energy Minimization," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1568-1583, Oct. 2006.
[20] J.K. Johnson, D.M. Malioutov, and A.S. Willsky, "Lagrangian Relaxation for MAP Estimation in Graphical Models," Proc. Allerton Conf. Comm., Control and Computing, 2007.
[21] N. Komodakis, N. Paragios, and G. Tziritas, "MRF Optimization via Dual Decomposition: Message-Passing Revisited," Proc. Int'l Conf. Computer Vision, 2007.
[22] D.P. Bertsekas, Nonlinear Programming. Athena Scientific, 1999.
[23] Y. Weiss, C. Yanover, and T. Meltzer, "MAP Estimation, Linear Programming and Belief Propagation with Convex Free Energies," Proc. Conf. Uncertainty in Artificial Intelligence, 2007.
[24] C. Rother, T. Minka, A. Blake, and V. Kolmogorov, "Cosegmentation of Image Pairs by Histogram Matching—Incorporating a Global Constraint into MRFs," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[25] A. Koster, C.P.M. van Hoesel, and A.W.J. Kolen, "The Partial Constraint Satisfaction Problem: Facets and Lifting Theorems," Operations Research Letters, vol. 23, nos. 3-5, pp. 89-97, 1998.
[26] D. Sontag and T. Jaakkola, "New Outer Bounds on the Marginal Polytope," Proc. Neural Information Processing Systems, 2007.
[27] M.P. Kumar and P.H.S. Torr, "Efficiently Solving Convex Relaxations for MAP Estimation," Proc. Int'l Conf. Machine Learning, pp. 680-687, 2008.
[28] N. Komodakis and N. Paragios, "Beyond Loose LP-relaxations: Optimizing MRFs by Repairing Cycles," Proc. European Conf. Computer Vision, 2008.
[29] D. Sontag, T. Meltzer, A. Globerson, T. Jaakkola, and Y. Weiss, "Tightening LP Relaxations for MAP Using Message Passing," Proc. Conf. Uncertainty in Artificial Intelligence, 2008.
[30] M.M. Deza and M. Laurent, Geometry of Cuts and Metrics. Springer, 1997.
[31] F. Barahona and A.R. Mahjoub, "On the Cut Polytope," Math. Programming, vol. 36, no. 2, pp. 157-173, 1986.
[32] S. Lauritzen, Graphical Models. Oxford Univ. Press, 1996.
[33] A. Mackworth, "Constraint Satisfaction," Encyclopaedia of Artificial Intelligence, pp. 285-292, John Wiley and Sons, 1991.
[34] D. Cohen and P. Jeavons, "The Complexity of Constraint Languages," Handbook of Constraint Programming, chapter 8, Elsevier, 2006.
[35] J.K. Pearson and P.G. Jeavons, "A Survey of Tractable Constraint Satisfaction Problems," Technical Report CSD-TR-97-15, Royal Holloway, Univ. of London, July 1997.
[36] C. Bessiere, "Constraint Propagation," Handbook of Constraint Programming, chapter 3, Elsevier, 2006.
[37] R. Debruyne and C. Bessière, "Domain Filtering Consistencies," J. Artificial Intelligence Research, vol. 14, pp. 205-230, 2001.
[38] E.C. Freuder, "A Sufficient Condition for Backtrack-Free Search," J. ACM, vol. 29, no. 1, pp. 24-32, 1982.
[39] T. Werner, "Marginal Consistency: Unifying Constraint Propagation on Commutative Semirings," Proc. Int'l Workshop Preferences and Soft Constraints (Co-Located with Conf. Principles and Practice of Constraint Programming), pp. 43-57, Sept. 2008.
[40] C. Bliek and D. Sam-Haroud, "Path Consistency on Triangulated Constraint Graphs," Proc. Int'l Joint Conf. Artificial Intelligence, pp. 456-461, 1999.
[41] M.I. Schlesinger, "False Minima of the Algorithm for Minimizing Energy of Max-Sum Labeling Problem," Glushkov Inst. of Cybernetics, unpublished, 1976.
[42] V.N. Kolmogorov and M.J. Wainwright, "On the Optimality of Tree-Reweighted Max-Product Message-Passing," Proc. Conf. Uncertainty in Artificial Intelligence, 2005.
[43] A. Globerson and T. Jaakkola, "Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations," Proc. Neural Information Processing Systems, pp. 553-560, 2008.
[44] T. Werner and A. Shekhovtsov, "Unified Framework for Semiring-Based Arc Consistency and Relaxation Labeling," Proc. 12th Computer Vision Winter Workshop, pp. 27-34, Feb. 2007.
[45] M.I. Schlesinger and V.V. Giginjak, "Solving (max,+) Problems of Structural Pattern Recognition Using Equivalent Transformations," Upravlyayushchie Sistemy i Mashiny (Control Systems and Machines), vols. 1/2, 2007 (in Russian), English translation available.
[46] P. Ravikumar, A. Agarwal, and M.J. Wainwright, "Message-Passing for Graph-Structured Linear Programs: Proximal Projections, Convergence and Rounding Schemes," Proc. Int'l Conf. Machine Learning, pp. 800-807, 2008.
[47] W.-J. van Hoeve and I. Katriel, "Global Constraints," Handbook of Constraint Programming, chapter 7, Elsevier, 2006.
[48] C. Bessière and P.V. Hentenryck, "To Be or Not to Be ... a Global Constraint," Proc. Conf. Principles and Practice of Constraint Programming, pp. 789-794, 2003.
[49] R. Gupta, A.A. Diwan, and S. Sarawagi, "Efficient Inference with Cardinality-Based Clique Potentials," Proc. Int'l Conf. Machine Learning, pp. 329-336, 2007.
[50] M.I. Schlesinger and B. Flach, "Some Solvable Subclasses of Structural Recognition Problems," Proc. Czech Pattern Recognition Workshop, 2000.
[51] D.M. Topkis, Supermodularity and Complementarity. Princeton Univ. Press, 1998.
[52] R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwal, M. Tappen, and C. Rother, "A Comparative Study of Energy Minimization Methods for Markov Random Fields," Proc. European Conf. Computer Vision, vol. II, pp. 16-29, 2006.
[53] V. Kolmogorov and C. Rother, "Comparison of Energy Minimization Algorithms for Highly Connected Graphs," Proc. European Conf. Computer Vision, vol. II, pp. 1-15, 2006.
[54] E. Gregoir, B. Mazure, and C. Piette, "MUST: Provide a Finer-Grained Explanation of Unsatisfiability," Proc. Conf. Principles and Practice of Constraint Programming, pp. 317-331, 2007.
[55] F. Hemery, C. Lecoutre, L. Sais, and F. Boussemart, "Extracting MUCs from Constraint Networks," Proc. European Conf. Artificial Intelligence, pp. 113-117, 2006.
[56] A. Globerson and T. Jaakkola, "Approximate Inference Using Planar Graph Decomposition," Proc. Neural Information Processing Systems, pp. 473-480, 2006.
[57] N.N. Schraudolph and D. Kamenetsky, "Efficient Exact Inference in Planar Ising Models," Proc. Neural Information Processing Systems, pp. 1417-1424, 2008.

Index Terms:
Weighted constraint satisfaction, Gibbs distribution, graphical model, Markov random field, linear programming relaxation, marginal polytope, cut polytope, cutting-plane algorithm, global constraint, supermodularity, tree-reweighted max-product.
Citation:
Tomáš Werner, "Revisiting the Linear Programming Relaxation Approach to Gibbs Energy Minimization and Weighted Constraint Satisfaction," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 8, pp. 1474-1488, Aug. 2010, doi:10.1109/TPAMI.2009.134
Usage of this product signifies your acceptance of the Terms of Use.