This Article 
 Bibliographic References 
 Add to: 
Combining Belief Networks and Neural Networks for Scene Segmentation
April 2002 (vol. 24 no. 4)
pp. 467-483

We are concerned with the problem of image segmentation, in which each pixel is assigned to one of a predefined finite number of labels. In Bayesian image analysis, this requires fusing together local predictions for the class labels with a prior model of label images. Following the work of, we consider the use of tree-structured belief networks (TSBNs) as prior models. The parameters in the TSBN are trained using a maximum-likelihood objective function with the EM algorithm and the resulting model is evaluated by calculating how efficiently it codes label images. A number of authors have used Gaussian mixture models to connect the label field to the image data. In this paper, we compare this approach to the scaled-likelihood method of where local predictions of pixel classification from neural networks are fused with the TSBN prior. Our results show a higher performance is obtained with the neural networks. We evaluate the classification results obtained and emphasize not only the maximum a posteriori segmentation, but also the uncertainty, as evidenced e.g., by the pixelwise posterior marginal entropies. We also investigate the use of conditional maximum-likelihood training for the TSBN and find that this gives rise to improved classification performance over the ML-trained TSBN.

[1] D.H. Ballard and C.M. Brown, Computer Vision, Prentice Hall, Upper Saddle River, N.J., 1982.
[2] M. Basseville, A. Benveniste, K.C. Chou, S.A. Golden, R. Nikoukhah, and A.S. Willsky, “Modeling and Estimation of Multiresolution Stochastic Processes,” IEEE Trans. Information Theory, vol. 38, no. 2, pp. 766-784, 1992.
[3] J. Besag, “On the Statistical Analysis of Dirty Pictures,” J. Royal Statistics, Soc. B, vol. 48, no. 3, pp. 259-302, 1974.
[4] C.M. Bishop, Neural Networks for Pattern Recognition. Clarendon Press, 1995.
[5] C.A. Bouman and M. Shapiro, “A Multiscale Random Field Model for Bayesian Image Segmentation,” IEEE Trans. Image Processing, vol. 3, pp. 162-176, 1994.
[6] H. Cheng and C.A. Bouman, “Trainable Context Model for Multiscale Segmentation,” Proc. Int'l Conf. Image Processing (ICIP '98), vol. 1, pp. 610-614, 1998.
[7] M.S. Crouse, R.D. Novak, and R.G. Baraniuk, “Wavelet-Based Statistical Signal Processing Using Hidden Markov Models,” IEEE Trans. Signal Processing, vol. 46, pp. 886-902, Apr. 1998.
[8] P. Dayan, G.E. Hinton, R.M. Neal, and R.S. Zemel, The Helmholtz Machine Neural Computation, vol. 7, no. 5, pp. 889-904, 1995.
[9] J.S. De Bonet and P. Viola, “A Non-Parametric Multi-Scale Statistical Model for Natural Images,” Advances in Neural Information Processing, vol. 10, 1997.
[10] J.A. Feldman and Y. Yakimovsky, “Decision Theory and Artificial Intelligence: I. A Semantics-Based Region Analyzer,” Artificial Intelligence, vol. 5, pp. 349-371, 1974.
[11] P. Fieguth, W. Karl, and A.S. Willsky, “Efficient Multiresolution Counterparts to Variational Methods for Surface Reconstruction,” Computer Vision and Image Understanding, vol. 70, no. 2, pp. 157-176, 1998.
[12] B.J. Frey and D.J.C. MacKay, “A Revolution: Belief Propagation in Graphs with Cycles,” Advances in Neural Information Processing Systems 10, M.I. Jordan, M.J. Kearns, and S.A. Solla, eds. Cambridge, Mass.: MIT Press, 1998.
[13] S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE Trans. Pattern Analysis and Machine Intellligence, vol. 6, no. 6, pp. 721-741, 1984.
[14] P.S. Gopalakrishnan, D. Kanevsky, A. Nadas, and D. Nahamoo, "An Inequality for Rational Functions With Applications to Some Statistical Estimation Problems," IEEE Trans. Information Theory, vol. 37, no. 1, pp. 107-113, 1991.
[15] F. Jensen, An Introduction to Bayesian Neworks. Springer Verlag, 1996.
[16] J. Kittler, “Statistical Pattern Recognition in Image Analysis,” Statistics and Images 2, K.V. Mardia, ed. Carfax Publishing Co. pp. 61-75, 1994.
[17] S. Krishnamachari and R. Chellappa, “Multiresolution Gauss-Markov Random Field Models for Texture Segmentation,” IEEE Trans. Image Processing, vol. 6, no. 2, pp. 251-267, 1997.
[18] A. Krogh, “Hidden Markov Models for Labeled Sequences,” Proc. 12th IAPR Int'l Conf. Pattern Recognition, pp. 140-144, 1994.
[19] J-M. Laferté, F. Heitz, P. Pérez, and E. Fabre, “Hierarchical Statistical Models for the Fusion of Multiresolution Data,” Proc. Fifth Int'l Conf. Computer Vision, 1995.
[20] J-M. Laferte, P. Perez, and F Heitz, “Discrete Markov Image Modeling and Inference on the Quadtree,” IEEE Trans. Image Processing, vol. 9, pp. 390-404, 2000.
[21] S.L. Lauritzen, Graphical Models. Oxford Univ. Press, 1996.
[22] M.D. Levine and A.M. Nazif, “Dynamic Measurement of Computer Generated Image Segmentations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 2, pp. 155-164, 1985.
[23] T.M. Lillesand and R.W. Kiefer, Remote Sensing and Image Interpretation, third ed. New York: Wiley, 1994.
[24] H. Lucke, Bayesian Belief Networks as a Tool for Stochastic Parsing Speech Comm., vol. 16, pp. 89-118, 1995.
[25] M.R. Luettgen, W. Karl, and A.S. Willsky, Efficient Multiscale Regularization with Applications to the Computation of Optical Flow IEEE Trans. Image Processing, vol. 3, pp. 41-64, 1994.
[26] M.R. Luettgen, A.S. Willsky, Likelihood Calculation for a Class of Multiscale Stochastic Models, with Application to Texture Discrimination IEEE Trans. Image Processing, vol. 4, no. 2, pp. 194-207, 1995.
[27] K.V. Mardia, A.J. Baczkowski, X. Feng, and T.J. Hainsworth, “Statistical Methods for Automatic Interpretation of Digitally Scanned Finger Prints,” Pattern Recognition Letters, vol. 18, pp. 1197-1203, 1997.
[28] J.D. McCauley and B.A. Engel, “Comparison of Scene Segmentations: SMAP, ECHO and Maximum Likelihood,” IEEE Trans. Geoscience and Remote Sensing, vol. 33, no. 6, pp. 1313-1316, 1995.
[29] W.A. McKeown, D.M. Harvey, and J. McDermott, “Rule-Based Interpretation of Aerial Imagery,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 5, pp. 570-585, 1985.
[30] M. Møller, “A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning,” Neural Networks, vol. 6, pp. 525–533, 1993.
[31] N. Morgan and H.A. Bourlard, “Neural Networks for Statistical Recognition of Continuous Speech,” Proc. IEEE, vol. 83, no. 5, pp. 742-770, 1995.
[32] R. Neal, Bayesian Learning for Neural Networks. New York: Springer Verlag, 1996.
[33] K.M. Nickels and S. Hutchinson, “Textured Image Segmentation: Returning Multiple Solutions,” Image and Vision Computing, vol. 15, pp. 781-795, 1997.
[34] Y. Ohta, Knowledge-Based Interpretation of Outdoor Natural ColorScenes.Massachusetts: Pitman Publishing, 1985.
[35] J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Mateo, Calif.: Morgan Kaufman, 1988.
[36] P. Pérez, A. Chardin, and J-M. Laferté, “Noniterative Manipulation of Discrete Energy-Based Models for Image Analysis,” Pattern Recognition, vol. 33, no. 4, pp. 573-586, 2000.
[37] L.R. Rabiner, “Tutorial on Hidden Markov Model and Selected Applications in Speech Recognition,” Proc. IEEE, vol. 77, no. 2, pp. 257-285, 1989.
[38] M.D. Richard and R.P. Lippmann, “Neural Network Classifiers Estimate Bayesian A-Posteriori Probabilities,” Neural Computation, vol. 3, no. 4, pp. 461-483, 1991.
[39] S.K. Riis and A. Krogh, “Hidden Neural Networks: A Framework for (HMM/NN) Hybrids,” Proc. Int'l Conf. Acoustics, Speech, and Signal Processing, (ICASSP-97), 1997.
[40] O. Ronen, J.R. Rohlicek, and M. Ostendorf, Parameter Estimation of Dependence Tree Models Using the EM Algorithm IEEE Signal Processing Letters, vol. 2, no. 8, pp. 157-159, 1995.
[41] L. Saul and M.I. Jordan, “A Mean Field Learning Algorithm for Unsupervised Neural Networks,” Learning in Graphical Models, M.I. Jordan, ed. Kluwer Academic, pp. 541-554, 1998.
[42] P. Smyth, “Hidden Markov Models for Fault Detection in Dynamic Systems,” Pattern Recogntion, vol. 27, no. 1, pp. 149-164, 1994.
[43] A. Storkey and C.K.I. Williams, “Dynamic Positional Trees for Structural Image Analysis,” Proc. Eighth Int'l Workshop Artificial Intelligence and Statistics, 2001.
[44] M.J. Weinberger, G. Seroussi, and G. Sapiro, "Loco-I: A Low Complexity, Context Based, Lossless Image Compression Algorithm," Proc. Data Compression Conf. 96, IEEE Computer Soc. Press, Los Alamitos, Calif., Apr. 1996, pp. 140-149.
[45] M. Weinberger, G. Seroussi, and G. Sapiro, “The LOCO-I Lossless Image Compression Algorithm: Principles and Standardization into JPEG-LS,” Hewlett-Packard Laboratories Technical Report No. HPL-98-193R1, Nov. 1998, revised, Oct. 1999, available fromhttp://www.hpl.hp.comloco/. To appear inIEEE Trans. Image Processing.
[46] Y. Weiss and W.T. Freeman, “Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology,” Advances in Neural Information Processing Systems 12, S.A. Solla, T.K. Leen, and K.-R. Müller, eds., Cambridge, Mass.: MIT Press, 2000.
[47] R.M. Welch, M.S. Navar, and S.K. Sengupta, “The Effect of Spatial Resolution Upon Texture-Based Cloud Field Classifications,” J. Geophysical Research, vol. 94, pp. 14767-14781, 1989.
[48] J.S. Weszka, C.R. Dyer, and A. Rosenfeld, “A Comparative Study of Texture Measures for Terrain Classification,” IEEE Trans. Systems, Man, and Cybernetics, vol. 6, no. 4, pp. 269-285, 1976.
[49] C.K.I. Williams and N.J. Adams, DTs: Dynamic Trees Proc. Advances in Neural Information Processing Systems, M.J. Kearns, S.A. Solla, and D.A. Cohn, eds., vol. 11, 1999.
[50] W.A. Wright, “Image Labeling with A Neural Network,” Proc. Fifth Alvey Vision Conf., pp. 227-232, 1989.
[51] W.A. Wright, W.P.J. Mackeown, and P. Greenway, “The Use of Neural Networks for Region Labeling and Scene Understanding,” Neural Networks, J.G Taylor, ed., pp. 165-192, 1995.

Index Terms:
tree-structured belief network (TSBN), hierarchical modeling, Markov random field (MRF), neural network, scaled-likelihood method, conditional maximum-likelihood training, Gaussian mixture model, expectation-maximization (EM)
X. Feng, C.K.I. Williams, S.N. Felderhof, "Combining Belief Networks and Neural Networks for Scene Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 467-483, April 2002, doi:10.1109/34.993555
Usage of this product signifies your acceptance of the Terms of Use.