This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Combining Belief Networks and Neural Networks for Scene Segmentation
April 2002 (vol. 24 no. 4)
pp. 467-483

We are concerned with the problem of image segmentation, in which each pixel is assigned to one of a predefined finite number of labels. In Bayesian image analysis, this requires fusing together local predictions for the class labels with a prior model of label images. Following the work of, we consider the use of tree-structured belief networks (TSBNs) as prior models. The parameters in the TSBN are trained using a maximum-likelihood objective function with the EM algorithm and the resulting model is evaluated by calculating how efficiently it codes label images. A number of authors have used Gaussian mixture models to connect the label field to the image data. In this paper, we compare this approach to the scaled-likelihood method of where local predictions of pixel classification from neural networks are fused with the TSBN prior. Our results show a higher performance is obtained with the neural networks. We evaluate the classification results obtained and emphasize not only the maximum a posteriori segmentation, but also the uncertainty, as evidenced e.g., by the pixelwise posterior marginal entropies. We also investigate the use of conditional maximum-likelihood training for the TSBN and find that this gives rise to improved classification performance over the ML-trained TSBN.

[1] D.H. Ballard and C.M. Brown, Computer Vision, Prentice Hall, Upper Saddle River, N.J., 1982.
[2] M. Basseville, A. Benveniste, K.C. Chou, S.A. Golden, R. Nikoukhah, and A.S. Willsky, “Modeling and Estimation of Multiresolution Stochastic Processes,” IEEE Trans. Information Theory, vol. 38, no. 2, pp. 766-784, 1992.
[3] J. Besag, “On the Statistical Analysis of Dirty Pictures,” J. Royal Statistics, Soc. B, vol. 48, no. 3, pp. 259-302, 1974.
[4] C.M. Bishop, Neural Networks for Pattern Recognition. Clarendon Press, 1995.
[5] C.A. Bouman and M. Shapiro, “A Multiscale Random Field Model for Bayesian Image Segmentation,” IEEE Trans. Image Processing, vol. 3, pp. 162-176, 1994.
[6] H. Cheng and C.A. Bouman, “Trainable Context Model for Multiscale Segmentation,” Proc. Int'l Conf. Image Processing (ICIP '98), vol. 1, pp. 610-614, 1998.
[7] M.S. Crouse, R.D. Novak, and R.G. Baraniuk, “Wavelet-Based Statistical Signal Processing Using Hidden Markov Models,” IEEE Trans. Signal Processing, vol. 46, pp. 886-902, Apr. 1998.
[8] P. Dayan, G.E. Hinton, R.M. Neal, and R.S. Zemel, The Helmholtz Machine Neural Computation, vol. 7, no. 5, pp. 889-904, 1995.
[9] J.S. De Bonet and P. Viola, “A Non-Parametric Multi-Scale Statistical Model for Natural Images,” Advances in Neural Information Processing, vol. 10, 1997.
[10] J.A. Feldman and Y. Yakimovsky, “Decision Theory and Artificial Intelligence: I. A Semantics-Based Region Analyzer,” Artificial Intelligence, vol. 5, pp. 349-371, 1974.
[11] P. Fieguth, W. Karl, and A.S. Willsky, “Efficient Multiresolution Counterparts to Variational Methods for Surface Reconstruction,” Computer Vision and Image Understanding, vol. 70, no. 2, pp. 157-176, 1998.
[12] B.J. Frey and D.J.C. MacKay, “A Revolution: Belief Propagation in Graphs with Cycles,” Advances in Neural Information Processing Systems 10, M.I. Jordan, M.J. Kearns, and S.A. Solla, eds. Cambridge, Mass.: MIT Press, 1998.
[13] S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE Trans. Pattern Analysis and Machine Intellligence, vol. 6, no. 6, pp. 721-741, 1984.
[14] P.S. Gopalakrishnan, D. Kanevsky, A. Nadas, and D. Nahamoo, "An Inequality for Rational Functions With Applications to Some Statistical Estimation Problems," IEEE Trans. Information Theory, vol. 37, no. 1, pp. 107-113, 1991.
[15] F. Jensen, An Introduction to Bayesian Neworks. Springer Verlag, 1996.
[16] J. Kittler, “Statistical Pattern Recognition in Image Analysis,” Statistics and Images 2, K.V. Mardia, ed. Carfax Publishing Co. pp. 61-75, 1994.
[17] S. Krishnamachari and R. Chellappa, “Multiresolution Gauss-Markov Random Field Models for Texture Segmentation,” IEEE Trans. Image Processing, vol. 6, no. 2, pp. 251-267, 1997.
[18] A. Krogh, “Hidden Markov Models for Labeled Sequences,” Proc. 12th IAPR Int'l Conf. Pattern Recognition, pp. 140-144, 1994.
[19] J-M. Laferté, F. Heitz, P. Pérez, and E. Fabre, “Hierarchical Statistical Models for the Fusion of Multiresolution Data,” Proc. Fifth Int'l Conf. Computer Vision, 1995.
[20] J-M. Laferte, P. Perez, and F Heitz, “Discrete Markov Image Modeling and Inference on the Quadtree,” IEEE Trans. Image Processing, vol. 9, pp. 390-404, 2000.
[21] S.L. Lauritzen, Graphical Models. Oxford Univ. Press, 1996.
[22] M.D. Levine and A.M. Nazif, “Dynamic Measurement of Computer Generated Image Segmentations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 2, pp. 155-164, 1985.
[23] T.M. Lillesand and R.W. Kiefer, Remote Sensing and Image Interpretation, third ed. New York: Wiley, 1994.
[24] H. Lucke, Bayesian Belief Networks as a Tool for Stochastic Parsing Speech Comm., vol. 16, pp. 89-118, 1995.
[25] M.R. Luettgen, W. Karl, and A.S. Willsky, Efficient Multiscale Regularization with Applications to the Computation of Optical Flow IEEE Trans. Image Processing, vol. 3, pp. 41-64, 1994.
[26] M.R. Luettgen, A.S. Willsky, Likelihood Calculation for a Class of Multiscale Stochastic Models, with Application to Texture Discrimination IEEE Trans. Image Processing, vol. 4, no. 2, pp. 194-207, 1995.
[27] K.V. Mardia, A.J. Baczkowski, X. Feng, and T.J. Hainsworth, “Statistical Methods for Automatic Interpretation of Digitally Scanned Finger Prints,” Pattern Recognition Letters, vol. 18, pp. 1197-1203, 1997.
[28] J.D. McCauley and B.A. Engel, “Comparison of Scene Segmentations: SMAP, ECHO and Maximum Likelihood,” IEEE Trans. Geoscience and Remote Sensing, vol. 33, no. 6, pp. 1313-1316, 1995.
[29] W.A. McKeown, D.M. Harvey, and J. McDermott, “Rule-Based Interpretation of Aerial Imagery,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 5, pp. 570-585, 1985.
[30] M. Møller, “A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning,” Neural Networks, vol. 6, pp. 525–533, 1993.
[31] N. Morgan and H.A. Bourlard, “Neural Networks for Statistical Recognition of Continuous Speech,” Proc. IEEE, vol. 83, no. 5, pp. 742-770, 1995.
[32] R. Neal, Bayesian Learning for Neural Networks. New York: Springer Verlag, 1996.
[33] K.M. Nickels and S. Hutchinson, “Textured Image Segmentation: Returning Multiple Solutions,” Image and Vision Computing, vol. 15, pp. 781-795, 1997.
[34] Y. Ohta, Knowledge-Based Interpretation of Outdoor Natural ColorScenes.Massachusetts: Pitman Publishing, 1985.
[35] J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Mateo, Calif.: Morgan Kaufman, 1988.
[36] P. Pérez, A. Chardin, and J-M. Laferté, “Noniterative Manipulation of Discrete Energy-Based Models for Image Analysis,” Pattern Recognition, vol. 33, no. 4, pp. 573-586, 2000.
[37] L.R. Rabiner, “Tutorial on Hidden Markov Model and Selected Applications in Speech Recognition,” Proc. IEEE, vol. 77, no. 2, pp. 257-285, 1989.
[38] M.D. Richard and R.P. Lippmann, “Neural Network Classifiers Estimate Bayesian A-Posteriori Probabilities,” Neural Computation, vol. 3, no. 4, pp. 461-483, 1991.
[39] S.K. Riis and A. Krogh, “Hidden Neural Networks: A Framework for (HMM/NN) Hybrids,” Proc. Int'l Conf. Acoustics, Speech, and Signal Processing, (ICASSP-97), 1997.
[40] O. Ronen, J.R. Rohlicek, and M. Ostendorf, Parameter Estimation of Dependence Tree Models Using the EM Algorithm IEEE Signal Processing Letters, vol. 2, no. 8, pp. 157-159, 1995.
[41] L. Saul and M.I. Jordan, “A Mean Field Learning Algorithm for Unsupervised Neural Networks,” Learning in Graphical Models, M.I. Jordan, ed. Kluwer Academic, pp. 541-554, 1998.
[42] P. Smyth, “Hidden Markov Models for Fault Detection in Dynamic Systems,” Pattern Recogntion, vol. 27, no. 1, pp. 149-164, 1994.
[43] A. Storkey and C.K.I. Williams, “Dynamic Positional Trees for Structural Image Analysis,” Proc. Eighth Int'l Workshop Artificial Intelligence and Statistics, 2001.
[44] M.J. Weinberger, G. Seroussi, and G. Sapiro, "Loco-I: A Low Complexity, Context Based, Lossless Image Compression Algorithm," Proc. Data Compression Conf. 96, IEEE Computer Soc. Press, Los Alamitos, Calif., Apr. 1996, pp. 140-149.
[45] M. Weinberger, G. Seroussi, and G. Sapiro, “The LOCO-I Lossless Image Compression Algorithm: Principles and Standardization into JPEG-LS,” Hewlett-Packard Laboratories Technical Report No. HPL-98-193R1, Nov. 1998, revised, Oct. 1999, available fromhttp://www.hpl.hp.comloco/. To appear inIEEE Trans. Image Processing.
[46] Y. Weiss and W.T. Freeman, “Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology,” Advances in Neural Information Processing Systems 12, S.A. Solla, T.K. Leen, and K.-R. Müller, eds., Cambridge, Mass.: MIT Press, 2000.
[47] R.M. Welch, M.S. Navar, and S.K. Sengupta, “The Effect of Spatial Resolution Upon Texture-Based Cloud Field Classifications,” J. Geophysical Research, vol. 94, pp. 14767-14781, 1989.
[48] J.S. Weszka, C.R. Dyer, and A. Rosenfeld, “A Comparative Study of Texture Measures for Terrain Classification,” IEEE Trans. Systems, Man, and Cybernetics, vol. 6, no. 4, pp. 269-285, 1976.
[49] C.K.I. Williams and N.J. Adams, DTs: Dynamic Trees Proc. Advances in Neural Information Processing Systems, M.J. Kearns, S.A. Solla, and D.A. Cohn, eds., vol. 11, 1999.
[50] W.A. Wright, “Image Labeling with A Neural Network,” Proc. Fifth Alvey Vision Conf., pp. 227-232, 1989.
[51] W.A. Wright, W.P.J. Mackeown, and P. Greenway, “The Use of Neural Networks for Region Labeling and Scene Understanding,” Neural Networks, J.G Taylor, ed., pp. 165-192, 1995.

Index Terms:
tree-structured belief network (TSBN), hierarchical modeling, Markov random field (MRF), neural network, scaled-likelihood method, conditional maximum-likelihood training, Gaussian mixture model, expectation-maximization (EM)
Citation:
X. Feng, C.K.I. Williams, S.N. Felderhof, "Combining Belief Networks and Neural Networks for Scene Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 467-483, April 2002, doi:10.1109/34.993555
Usage of this product signifies your acceptance of the Terms of Use.