The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - October (2010 vol.32)
pp: 1744-1757
Zhuowen Tu , University of California at Los Angeles, Los Angeles
Xiang Bai , Huazhong University of Science and Technology, Wuhan
ABSTRACT
The notion of using context information for solving high-level vision and medical image segmentation problems has been increasingly realized in the field. However, how to learn an effective and efficient context model, together with an image appearance model, remains mostly unknown. The current literature using Markov Random Fields (MRFs) and Conditional Random Fields (CRFs) often involves specific algorithm design in which the modeling and computing stages are studied in isolation. In this paper, we propose a learning algorithm, auto-context. Given a set of training images and their corresponding label maps, we first learn a classifier on local image patches. The discriminative probability (or classification confidence) maps created by the learned classifier are then used as context information, in addition to the original image patches, to train a new classifier. The algorithm then iterates until convergence. Auto-context integrates low-level and context information by fusing a large number of low-level appearance features with context and implicit shape information. The resulting discriminative algorithm is general and easy to implement. Under nearly the same parameter settings in training, we apply the algorithm to three challenging vision applications: foreground/background segregation, human body configuration estimation, and scene region labeling. Moreover, context also plays a very important role in medical/brain images where the anatomical structures are mostly constrained to relatively fixed positions. With only some slight changes resulting from using 3D instead of 2D features, the auto-context algorithm applied to brain MRI image segmentation is shown to outperform state-of-the-art algorithms specifically designed for this domain. Furthermore, the scope of the proposed algorithm goes beyond image analysis and it has the potential to be used for a wide variety of problems for structured prediction problems.
INDEX TERMS
Context, object recognition, image segmentation, 3D brain segmentation, discriminative models, conditional random fields.
CITATION
Zhuowen Tu, Xiang Bai, "Auto-Context and Its Application to High-Level Vision Tasks and 3D Brain Image Segmentation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 10, pp. 1744-1757, October 2010, doi:10.1109/TPAMI.2009.186
REFERENCES
[1] S. Avidan, "Spatialboost: Adding Spatial Reasoning to Adaboost," Proc. European Conf. Computer Vision, pp. 386-396, 2006.
[2] S. Belongie, J. Malik, and J. Puzicha, "Shape Matching and Object Recognition Using Shape Contexts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 509-522, Apr. 2002.
[3] E. Borenstein, E. Sharon, and S. Ullman, "Combining Top-Down and Bottom-Up Segmentation," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshop Perceptual Organization in Computer Vision, June 2004.
[4] L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone, Classification and Regression Trees. Wadsworth Int'l, 1984.
[5] R. Caruana and A. Niculescu-Mizil, "An Empirical Comparison of Supervised Learning Algorithms," Proc. Int'l Conf. Machine Learning, pp. 161-168, 2006.
[6] C.C. Chang and C.J. Lin, LIBSVM: A Library for Support Vector Machines. http://www.csie.ntu.edu.tw/cjlinlibsvm, 2001.
[7] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 886-893, June 2005.
[8] T.G. Dietterich and G. Bakiri, "Solving Multiclass Learning Problems via Error-Correcting Output Codes," J. Artificial Intelligence Research, vol. 2, pp. 263-286, 1995.
[9] M. Fink and P. Perona, "Mutual Boosting for Contextual Inference," Proc. Neural Information Processing Systems Conf., 2003.
[10] B. Fischl, D. Salat, E. Busa, M. Albert, M. Dieterich, C. Haselgrove, A. van der Kouwe, R. Killiany, S.K.D. Kennedy, A. Montillo, N. Makris, B. Rosen, and A. Dale, "Whole Brain Segmentation: Automated Labeling of Neuroanatomical Structures in the Human Brain," Neuron, vol. 33, pp. 341-355, 2002.
[11] Y. Freund and R.E. Schapire, "A Decision-Theoretic Generalization of Online Learning and an Application to Boosting," J. Computer and System Sciences, vol. 55, no. 1, pp. 119-139, 1997.
[12] J. Friedman, T. Hastie, and R. Tibshirani, "Additive Logistic Regression: A Statistical View of Boosting," Annals of Statistics, vol. 38, no. 2, pp. 337-407, 2000.
[13] S. Geman and D. Geman, "Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 6, pp. 721-741, Nov. 1984.
[14] G. Gerig, M. Chakos, and M. Valmet, "A New Validation Tool for Assessing and Improving 3D Object Segmentation," Proc. Medical Image Computing and Computer-Assisted Intervention, pp. 516-523, 2001.
[15] X. He, R. Zemel, and M. Carreira-Perpinan, "Multiscale Conditional Random Fields for Image Labelling," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 695-702, June 2004.
[16] D. Hoiem, A. Efros, and M. Hebert, "Geometric Context from a Single Image," Proc. IEEE Int'l Conf. Computer Vision, pp. 654-661, June 2005.
[17] D. Hoiem, A. Efros, and M. Hebert, "Closing the Loop on Scene Interpretation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.
[18] D. Hoiem, A. Efros, and M. Hebert, "Putting Objects in Perspective," Int'l J. Computer Vision, vol. 80, no. 1, pp. 3-15, Oct. 2008.
[19] J. Jiang and Z. Tu, "Efficient Scale Space Auto-Context for Image Segmentation and Labeling," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[20] R. Kassel, "A Comparison of Approaches to On-Line Handwritten Character Recognition," PhD thesis, MIT Spoken Language Systems Group, Massachusetts Inst. of Tech nology, 1995.
[21] S. Kumar and M. Hebert, "Discriminative Random Fields: A Discriminative Framework for Contextual Interaction in Classification," Proc. IEEE Int'l Conf. Computer Vision, pp. 1150-1159, Oct. 2003.
[22] S. Kumar and M. Hebert, "A Hierarchical Field Framework for Unified Context-Based Classification," Proc. IEEE Int'l Conf. Computer Vision, pp. 1284-1291, Oct. 2005.
[23] J. Lafferty, A. McCallum, and F. Pereira, "Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data," Proc. 10th Int'l Conf. Machine Learning, pp. 282-289, 2001.
[24] Z. Lao, D. Shen, A. Jawad, B. Karacali, D. Liu, E. Melhem, N. Bryan, and C. Davatzikos, "Automated Segmentation of White Matter Lesions in 3D Brain MR Images, Using Multivariate Pattern Classification," Proc. Third IEEE Int'l Symp. Biomedical Imaging, pp. 307-310, Apr. 2006.
[25] C.B. Liu, A. Toga, and Z. Tu, "Fusing Adaptive Atlas and Informative Features for Robust 3D Brain Image Segmentation," technical report, Lab of Neuro Imaging, Univ. of California, Los Angeles, 2009.
[26] G. Mori, X. Ren, A. Efros, and J. Malik, "Recovering Human Body Configurations: Combining Segmentation and Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 326-333, June 2004.
[27] K.L. Narr, P.M. Thompson, T. Sharma, J. Moussai, R. Blanton, B. Anvar, A. Edris, R. Krupp, J. Rayman, M. Khaledy, and A.W. Toga, "Three-Dimensional Mapping of Temporo-Limbic Regions and the Lateral Ventricles in Schizophrenia: Gender Effects," Biological Psychiatry, vol. 50, no. 2, pp. 84-97, 2001.
[28] A. Oliva and A. Torralba, "The Role of Context in Object Recognition," Trends in Cognitive Sciences, vol. 11, no. 12, pp. 520-527, Dec. 2007.
[29] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
[30] S. Pizer, T. Fletcher, Y. Fridman, D. Fritsch, A. Gash, J. Glotzer, S. Joshi, A. Thall, G. Tracton, P. Yushkevich, and E. Chaney, "Deformable m-Reps for 3D Medical Image Segmentation," Int'l J. Computer Vision, vol. 55, no. 2, pp. 85-106, 2003.
[31] K. Pohl, J. Fisher, R. Kikinis, W. Grimson, and W. Wells, "A Bayesian Model for Joint Segmentation and Registration," NeuroImage, vol. 31, no. 1, pp. 228-239, 2006.
[32] A. Rabinovich, A. Vedaldi, C. Galleguillos, E. Wiewiora, and S. Belongie, "Objects in Context," Proc. IEEE Int'l Conf. Computer Vision, Oct. 2007.
[33] X. Ren, C. Fowlkes, and J. Malik, "Cue Integration in Figure/Ground Labeling," Proc. Neural Information Processing Systems Conf., 2005.
[34] R. Rifkin and A. Klautau, "In Defence of One-vs-All Classification," J. Machine Learning Research, vol. 5, pp. 101-141, 2004.
[35] T. Rohlfing, D.B. Russakoff, and J.C.R. Maurer, "Performance-Based Classifier Combination in Atlas-Based Image Segmentation Using Expectation-Maximization Parameter Estimation," IEEE Trans. Medical Imaging, vol. 23, no. 8, pp. 983-994, Aug. 2004.
[36] R.E. Schapire, R.E. Freund, P. Bartlett, and W.S. Lee, "Boosting the Margin: A New Explanation for the Effectiveness of Voting Methods," Annals of Statistics, vol. 26, pp. 1651-1686, 1998.
[37] J. Shotton, M. Johnson, and R. Cipolla, "Semantic Texton Forests for Image Categorization and Segmentation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[38] J. Shotton, J. Winn, C. Rother, and A. Criminisi, "TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-Class Object Recognition and Segmentation," Proc. European Conf. Computer Vision, pp. 1-15, 2006.
[39] A. Singhal, J. Luo, and W. Zhu, "Probabilistic Spatial Context Models for Scene Content Understanding," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2003.
[40] S. Smith, "Fast Robust Automated Brain Extraction," Human Brain Mapping, vol. 17, no. 3, pp. 856-876, 2001.
[41] R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. Tappen, and C. Rother, "A Comparative Study of Energy Minimization Methods for Markov Random Fields," Proc. European Conf. Computer Vision, 2006.
[42] B. Taskar, C. Guestrin, and D. Koller, "Max-Margin Markov Networks," Proc. Neural Information Processing Systems Conf., 2003.
[43] A. Torralba, K.P. Murphy, and W.T. Freeman, "Contextual Models for Object Detection Using Boosted Random Fields," Proc. Neural Information Processing Systems Conf., 2004.
[44] Z. Tu, "Probabilistic Boosting Tree: Learning Discriminative Models for Classification, Recognition, and Clustering," Proc. IEEE Int'l Conf. Computer Vision, pp. 1589-1596, Oct. 2005.
[45] Z. Tu, X. Chen, A. Yuille, and S. Zhu, "Image Parsing: Unifying Segmentation, Detection, and Object Recognition," Int'l J. Computer Vision, vol. 63, no. 2, pp. 113-140, July 2005.
[46] Z. Tu, K. Narr, P. Dollar, P. Thompson, and A. Toga, "Brain Anatomical Structure Parsing by Hybrid Discriminative/Generative Models," IEEE Trans. Medical Imaging, vol. 27, no. 4, pp. 495-508, Apr. 2008.
[47] B. van Ginneken, T. Heimann, and M. Styner, "3D Segmentation in the Clinic: A Grand Challenge," Proc. Medical Image Computing and Computer-Assisted Intervention Workshop, 2007.
[48] V. Vapnik, Estimation of Dependences Based on Empirical Data. Springer-Verlag, 1982.
[49] J. Verbeek and B. Triggs, "Region Classification with Markov Field Aspect Models," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2007.
[50] P.A. Viola and M.J. Jones, "Robust Real-Time Face Detection," Int'l J. Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[51] L. Wolf and S. Bileschi, "A Critical View of Context," Int'l J. Computer Vision, vol. 69, pp. 251-261, 2006.
[52] R.P. Woods, J.C. Mazziotta, and S.R. Cherry, "MRI-PET Registration with Automated Algorithm," J. Computer Assisted Tomography, vol. 17, pp. 536-546, 1993.
[53] Y. Wu and J. Fan, "Contextual Flow," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[54] J. Yang, L.H. Staib, and J.S. Duncan, "Neighbor-Constrained Segmentation with Level Set Based 3D Deformable Models," IEEE Trans. Medical Imaging, vol. 23, no. 8, pp. 940-948, Aug. 2004.
[55] M. Yang, G. Hua, and Y. Wu, "Context-Aware Visual Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 7, pp. 1195-1209, July 2009.
[56] J. Yedidia, W. Freeman, and Y. Weiss, "Generalized Belief Propagation," Proc. Neural Information Processing Systems Conf., 2000.
21 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool