This Article 
 Bibliographic References 
 Add to: 
Cluster Expansions for the Deterministic Computation of Bayesian Estimators Based on Markov Random Fields
March 1995 (vol. 17 no. 3)
pp. 275-293

Abstract—We describe a family of approximations, denoted by “cluster approximations,” for the computation of the mean of a Markov random field (MRF). This is a key computation in image processing when applied to the a posteriori MRF. The approximation is to account exactly for only spatially local interactions. Application of the approximation requires the solution of a nonlinear multivariable fixed-point equation for which we prove several existence, uniqueness, and convergence-of-algorithm results. Four numerical examples are presented, including comparison with Monte Carlo calculations.

[1] J. Besag,“Spatial interaction and the statistical analysis of lattice systems,” J. Royal Stat. Soc. B, vol. 36, no. 2, pp. 192-236, 1974.
[2] S. Geman and D. Geman,“Stochastic relaxation, Gibbs distributions, and the Bayesian restorationof images,” IEEE Trans. PAMI, vol. 6, no. 6, pp. 721-741, Nov. 1984.
[3] J. Marroquin,S. Mitter,, and T. Poggio,“Probabilistic solution of ill-posed problems in computational vision,” J. Am. Stat. Assoc., vol. 82, no. 397, pp. 76-89, 1987.
[4] J.W. Woods, “Two-Dimensional Discrete Markovian Fields,” IEEE Trans. Information Theory, vol. 18, pp. 232-240, 1972.
[5] J.W. Woods,“Markov image modeling,” IEEE Trans. Auto. Contr., vol. 23, no. 5, pp. 846-850, October 1978.
[6] R. Chellappa and R.L. Kashyap,“Digital image restoration using spatial interaction models,” IEEE Trans. ASSP, vol. 30, no. 3, pp. 461-472, June 1982.
[7] R.L. Kashyap and R. Chellappa,“Estimation and choice of neighbors in spatial-interaction models of images,” IEEE Trans. Information Theory, vol. 29, no. 1, pp. 60-72, Jan. 1983.
[8] P. Levy,“A special problem of Brownian motion, and a general theory of Gaussianrandom functions,” Proc. 3rd Berkeley Symp. Mathematical Statistics and Probability,Berkeley, CA, vol. 2, Univ. California Press, 1956.
[9] Y.A. Rosanov,“On Gaussian fields with given conditional distributions,” Theory Prob. Appl., vol. 12, pp. 381-391, 1967.
[10] J.W. Woods and C.H. Radewan,“Kalman filtering in two dimensions,” IEEE Trans. Info. Theory, vol. 23, pp. 232-240, Mar. 1972.
[11] J. Besag,“Nearest-neighbor systems and the auto-logistic model for binary data,” J. Royal Stat. Soc. B, vol. 34, no. 2, pp. 75-83, 1972.
[12] J. Besag,“On the statistical analysis of dirty pictures,” J. Royal Stat. Soc. B, vol. 48, pp. 259-302, 1986.
[13] E. Ising,Z. Physik, , vol. 31, pp. 253-258, 1925.
[14] H. Derin,H. Elliott,R. Cristi,, and D. Geman,“Bayes smoothing algorithms for segmentation of binary images modeled byMarkov random fields,” IEEE Trans. PAMI, vol. 6, no. 6, pp. 707-720, 1984.
[15] P.A. Devijver,“Segmentation of binary images using third-order Markov mesh image models,” Proc. 8th Int’l. Conf. Pattern Recognition, pp. 259-261,Paris, France, Oct. 1986.
[16] P.A. Devijver and M.M. Dekesel,“Learning the parameters of a hidden Markov random field image model: A simple example,” in Pattern Recognition Theory and Applications, P.A. Devijver and J. Kittler, Eds., pp. 141-163. Springer-Verlag, 1987.
[17] A.K. Jain,“Advances in mathematical models for image processing,” Proc. IEEE, vol. 69, no. 5, pp. 512-528, 1981.
[18] F.C. Jeng and J.W. Woods, "Compound Gauss-Markov Random Fields for Image Estimation," IEEE Trans. Signal Processing, vol. 39, pp. 683-697, 1991.
[19] F.-C. Jeng and J.W. Woods,“Simulated annealing in compound Gaussian random fields,” IEEE Trans. Info. Theory, vol. 36, no. 1, pp. 94-107, January 1990.
[20] F.-C. Jeng,J.W. Woods,, and S. Rastogi,“Compound Gauss-Markov Random Fields for Parallel Image Processing,” Chapter 2, Markov Random Fields: Theory and Application, Academic Press, Boston, 1993.
[21] J.W. Woods,S. Dravida,, and R. Mediavilla,“Image estimation using doubly stochastic Gaussian random field models,” IEEE Trans. PAMI, vol. 9, no. 2, pp. 245-253, Mar. 1987.
[22] S.E. Levinson,L.R. Rabiner,, and M.M. Sondhi,“An introduction to the application of the theory of probabilisticfunctions of a Markov process to automatic speech recognition,” BSTJ, vol. 62, no. 4, pp. 1035-1074, Apr. 1983.
[23] L.R. Rabiner,S.E. Levinson,, and M.M. Sondhi,“On the application of vector quantization and hidden Markov models tospeaker-independent, isolated word recognition,” BSTJ, vol. 62, no. 4, pp. 1075-1105, Apr. 1983.
[24] D. Geman,S. Geman,C. Graffigne,, and P. Dong,“Boundary detection by constrained optimization,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 7, pp. 609-628, July 1990.
[25] P.C. Doerschuk,“Bayesian signal reconstruction, Markov random fields, and x-raycrystallography,” J. Opt. Soc. Am. A, vol. 8, no. 8, pp. 1207-1221, 1991.
[26] P.C. Doerschuk,“Adaptive Bayesian signal reconstruction with a priori modelimplementation and synthetic examples for x-ray crystallography,” J. Opt. Soc. Am. A, vol. 8, no. 8, pp. 1222-1232, 1991.
[27] J. Biemond, R.L. Lagendijk, and R.M. Mersereau, "Iterative Methods for Image Deblurring," Proc. IEEE, vol. 78, pp. 856-883, 1990.
[28] G. Parisi,Statistical Field Theory, Section 3.2, Addison-Wesley, Redwood City, CA, 1988.
[29] G.L. Bilbro, W.E. Snyder, S.J. Garnier, and J.W. Gault, "Mean Field Annealing: A Formalism for Constructing GNC-Like Algorithms," IEEE Trans. Neural Networks, vol. 3, Jan. 1992.
[30] J.J. Clark and A.L. Yuille,Data Fusion for Sensory Information Processing Systems, Kluwer Academic, Boston, 1990.
[31] I.M. Elfadel and A.L. Yuille,“Mean-field theory for grayscale texture synthesis using Gibbs randomfields,” Stochastic and Neural Methods in Signal Processing, Image Processing, andComputer Vision, Su-Shing Chen, Ed., SPIE—The International Society for Optical Engineering, pp. 248-259, vol. 1569, San Diego, CA, July24-26 1991.
[32] I.M. Elfadel and A.L. Yuille,“Mean-field phase transitions for Gibbs random fields,” Stochastic and Neural Methods in Image and Signal Processing, Su-Shing Chen, Ed., SPIE—The International Society for Optical Engineering, pp. 257-268, vol. 1766, San Diego, CA, July20-23 1992.
[33] D. Geiger and F. Girosi,“Parallel and deterministic algorithms from MRFs: Surface reconstruction,” IEEE Transactions on PAMI, vol. 13, no. 5, pp. 401-412, May 1991.
[34] H.P. Hiriyannaiah,G.L. Bilbro,W.E. Snyder,, and R.C. Mann,“Restoration of piecewise-constant images by mean-field annealing,” J. Opt. Soc. Am. A, vol. 6, no. 12, pp. 1901-1912, 1989.
[35] J. Zhang,“The mean field theory in EM procedures for Markov random fields,” IEEE Transactions on Sig. Proc., vol. 40, no. 10, pp. 2570-2583, Oct. 1992.
[36] J. Zhang,“The mean field theory in EM procedures for blind Markov random field image restoration,” IEEE Transactions on Image Proc., vol. 2, no. 1, pp. 27-40, Jan. 1993.
[37] J.M. Ortega and W.C. Rheinboldt,Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, Inc., San Diego, CA, 1970.
[38] C.-h. Wu,“Deterministic Parallelizable Solutions for Bayesian Markov Random FieldEstimation Problems,” PhD thesis, Purdue University, West Lafayette, IN, May 1994.
[39] E.L. Allgower and K. Georg, Numerical Continuation Methods: An Introduction. New York: Springer 1990.
[40] A. Lumsdaine,J.L. Wyatt,, and I.M. Elfadel,“Nonlinear analog networks for image smoothing and segmentation,” J. VLSI Signal Processing, vol. 3, pp. 53-68, 1991.
[41] W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling, Numerical Recipes in C.Cambridge, England: Cambridge Univ. Press, 1988.
[42] R.K. Pathria,Statistical Mechanics, Fig. 12.23, Pergamon Press, Oxford, 1972.
[43] A. Blake and A. Zisserman, Visual Reconstruction. MIT Press, 1987.
[44] A. Senior,“Cursive handwriting image,” Ftp site, File pub/data/handwriting_page_image.tar.gz, 1994, Cambridge Univ. Eng. Dept, Trumpington Street, Cambridge CB2 1PZ ENGLAND; Phone: +44 (223) 3 32754 Fax: +44 (223) 3 32662.
[45] T. Simchony,R. Chellappa,, and Z. Lichtenstein,“Relaxation algorithms for MAP estimation of gray-level images withmultiplicative noise,” IEEE Trans. Info. Theory, vol. 36, no. 3, pp. 608-613, May 1990.

Index Terms:
Markov random fields, image restoration, Bayesian estimation, thresholded posterior mean estimator.
Chi-hsin Wu, Peter C. Doerschuk, "Cluster Expansions for the Deterministic Computation of Bayesian Estimators Based on Markov Random Fields," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 3, pp. 275-293, March 1995, doi:10.1109/34.368192
Usage of this product signifies your acceptance of the Terms of Use.