This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
C^4: Exploring Multiple Solutions in Graphical Models by Cluster Sampling
September 2011 (vol. 33 no. 9)
pp. 1713-1727
ASCII Text
x 
 
Jake Porway, Song-Chun Zhu, "C^4: Exploring Multiple Solutions in Graphical Models by Cluster Sampling," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 9, pp. 1713-1727, September, 2011.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2011.27,
author = {Jake Porway and Song-Chun Zhu},
title = {C^4: Exploring Multiple Solutions in Graphical Models by Cluster Sampling},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {33},
number = {9},
issn = {0162-8828},
year = {2011},
pages = {1713-1727},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2011.27},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - C^4: Exploring Multiple Solutions in Graphical Models by Cluster Sampling
IS - 9
SN - 0162-8828
SP1713
EP1727
EPD - 1713-1727
A1 - Jake Porway,
A1 - Song-Chun Zhu,
PY - 2011
KW - Markov random fields
KW - computer vision
KW - graph labeling
KW - probabilistic algorithms
KW - constraint satisfaction
KW - Monte Carlo.
VL - 33
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Jake Porway, R&D Division of The New York Times
Song-Chun Zhu, University of California, Los Angeles (UCLA) and the Lotus Hill Research Institute
This paper presents a novel Markov Chain Monte Carlo (MCMC) inference algorithm called C^4—Clustering with Cooperative and Competitive Constraints—for computing multiple solutions from posterior probabilities defined on graphical models, including Markov random fields (MRF), conditional random fields (CRF), and hierarchical models. The graphs may have both positive and negative edges for cooperative and competitive constraints. C^4 is a probabilistic clustering algorithm in the spirit of Swendsen-Wang [34]. By turning the positive edges on/off probabilistically, C^4 partitions the graph into a number of connected components (ccps) and each ccp is a coupled subsolution with nodes connected by positive edges. Then, by turning the negative edges on/off probabilistically, C^4 obtains composite ccps (called cccps) with competing ccps connected by negative edges. At each step, C^4 flips the labels of all nodes in a cccp so that nodes in each ccp keep the same label while different ccps are assigned different labels to observe both positive and negative constraints. Thus, the algorithm can jump between multiple competing solutions (or modes of the posterior probability) in a single or a few steps. It computes multiple distinct solutions to preserve the intrinsic ambiguities and avoids premature commitments to a single solution that may not be valid given later context. C^4 achieves a mixing rate faster than existing MCMC methods, such as various Gibbs samplers [15], [26] and Swendsen-Wang cuts [2], [34]. It is also more “dynamic” than common optimization methods such as ICM [3], LBP [21], [37], and graph cuts [4], [20]. We demonstrate the C^4 algorithm in line drawing interpretation, scene labeling, and object recognition.

[1] K.R. Apt, "The Essence of Constraint Propagation," Theoretical Computer Science, vol. 221, pp. 179-210, 1999.
[2] A. Barbu and S.C. Zhu, "Generalizing Swendsen-Wang to Sampling Arbitrary Posterior Probabilities," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1239-1253, Aug. 2005.
[3] J. Besag, "On the Statistical Analysis of Dirty Pictures," J. Royal Statistical Soc. Series B, vol. 48, no. 3, pp. 259-302, 1986.
[4] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[5] A. Braunstein, M. Mzard, and R. Zecchina, "Survey Propagation: An Algorithm for Satisfiability," Random Structures and Algorithms, vol. 27, pp. 201-226, 2005.
[6] H. Chui and A. Rangarajan, "A New Point Matching Algorithm for Non-Rigid Registration," Computer Vision and Image Understanding, vol. 89, no. 2, pp. 114-141, 2003.
[7] C. Cooper and A. Frieze, "Mixing Properties of the Swendsen-Wang Process in Classes of Graphs," Random Structures and Algorithms, vol. 15, nos. 3/4, pp. 242-261, 1999.
[8] T. Cormen, C.E. Leiserson, R.L. Rivest, and C. Stein, Introduction to Algorithms, second ed. MIT Press/McGraw-Hill, 2001.
[9] F. Dellaert, S. Seitz, C. Thorpe, and S. Thrun, "Feature Correspondence: A Markov Chain Monte Carlo Approach," Advances in Neural Information Processing Systems, vol. 13, MIT Press, pp. 852-858, 2001.
[10] R. Edwards and A. Sokal, "Generalization of the Fortuin-Kasteleyn-Swendsen-Wang Representation and Monte Carlo Algorithm," Physical Rev. Letters, vol. 38, pp. 2009-2012, 1988.
[11] P.F. Felzenszwalb and J.D. Schwartz, "Hierarchical Matching of Deformable Shapes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[12] R. Fergus, P. Perona, and A. Zisserman, "A Sparse Object Category Model for Efficient Learning and Exhaustive Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[13] R. Fletcher, "A New Approach to Variable Metric Algorithms," Computer J., vol. 13, pp. 317-322, 1970.
[14] A. Gelman, J.B. Carlin, H.S. Stern, and D.B. Rubin, Bayesian Data Analysis. second ed, chapter 5. Chapman and Hall/CRC, 2004.
[15] S. Geman and D. Geman, "Stochastic Relaxation, Gibbs Distributions and the Bayesian Restoration of Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 6, pp. 721-741, Nov. 1984.
[16] V.K. Gore and M.R. Jerrum, "The Swendsen-Wang Process Does Not Always Mix Rapidly," Proc. 29th Ann. ACM Symp. Theory of Computing, pp. 674-681, 1997.
[17] P. Green, "Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination," Biometrika, vol. 82, pp. 711-732, 1995.
[18] U. Grenander and M.I. Miller, "Representations of Knowledge in Complex Systems," J. Royal Statistical Soc. Series B, vol. 56, no. 4, pp. 549-603, 1994.
[19] D.A. Huffman, "Impossible Objects as Nonsense Sentences," Machine Intelligence, vol. 8, pp. 475-492, 1971.
[20] V. Kolmogorov and C. Rother, "Minimizing Nonsubmodular Functions with Graph Cuts-A Review," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 7, pp. 1274-1279, July 2007.
[21] M. Kumar and P. Torr, "Fast Memory-Efficient Generalized Belief Propagation," Lecture Notes in Computer Science, Springer-Verlag, 2006.
[22] S. Kumar and M. Hebert, "Man-Made Structure Detection in Natural Images Using a Causal Multiscale Random Field," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[23] J. Lafferty and F. Pereira, "Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data," Proc. Int'l Conf. Machine Learning, 2001.
[24] L. Lin, K. Zeng, X.B. Liu, and S.C. Zhu, "Layered Graph Matching by Composite Clustering with Collaborative and Competitive Interactions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[25] J. Liu, Monte Carlo Strategies in Scientific Computing. Springer, 2001.
[26] J. Liu, W.H. Wong, and A. Kong, "Correlation Structure and Convergence Rate of the Gibbs Sampler with Various Scans," J. Royal Statistical Soc. Series B, vol. 57, pp. 157-169, 1995.
[27] A.K. Mackworth, "Interpreting Pictures of Polyhedral Scenes," Artificial Intelligence, vol. 4, no. 2, pp. 121-137, 1973.
[28] A.K. Mackworth, "Consistency in Networks of Relations," Artificial Intelligence, vol. 8, pp. 99-118, 1977.
[29] S. Oh, J. Rehg, T. Balch, and F. Dellaert, "Learning and Inference in Parametric Switching Linear Dynamical Systems," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1161-1168, 2005.
[30] J. Pearl, Heuristics:Intelligent Search Strategies for Computer Problem Solving, Addison-Wesley Longman Publishing, 1984.
[31] J. Porway, K. Wang, and S.C. Zhu, "A Hierarchical and Contextual Model for Aerial Image Understanding," Int'l J. Computer Vision, vol. 88, no. 2, pp. 254-283, 2010.
[32] A. Rosenfeld, R.A. Hummel, and S.W. Zucker, "Scene Labeling by Relaxation Operations," IEEE Trans. Systems, Man, and Cybernetics, vol. 6, no. 6, pp. 420-433, June 1976.
[33] K. Sugihara, Machine Interpretation of Line Drawings. MIT Press, 1986.
[34] R.H. Swendsen and J.S. Wang, "Nonuniversal Critical Dynamics in Monte Carlo Simulations," Physical Rev. Letters, vol. 58, no. 2, pp. 86-88, 1987.
[35] Z. Tu and S.C. Zhu, "Image Segmentation by Data-Driven Markov Chain Monte Carlo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 657-673, May 2002.
[36] A. Torralba, K. Murphy, and W. Freeman, "Object Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[37] Y. Weiss, "Correctness of Local Probability Propagation in Graphical Models with Loops," Neural Computation, vol. 12, no. 1, pp. 1-41, 2000.
[38] B. Yao, M. Yang, and S.C. Zhu, "Introduction to a Large-Scale General Purpose Ground Truth Database: Methodology, Annotation Tools and Benchmarks," Proc. Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, 2007.
[39] T.F. Wu and S.C. Zhu, "A Numeric Study of the Bottom-Up and Top-Down Inference Processes in And-Or Graphs," Int'l J. Computer Vision, 2010.
[40] S.C. Zhu and D. Mumford, "A Stochastic Grammar of Images," Foundations and Trends in Computer Graphics and Vision, vol. 2, no. 4, pp. 259-362, 2006.

Index Terms:
Markov random fields, computer vision, graph labeling, probabilistic algorithms, constraint satisfaction, Monte Carlo.
Citation:
Jake Porway, Song-Chun Zhu, "C^4: Exploring Multiple Solutions in Graphical Models by Cluster Sampling," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 9, pp. 1713-1727, Sept. 2011, doi:10.1109/TPAMI.2011.27
Usage of this product signifies your acceptance of the Terms of Use.