The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - April (2009 vol.31)
pp: 661-676
Michael G. Ross , Massachusetts Institute of Technology, Cambridge
ABSTRACT
The Segmentation According to Natural Examples (SANE) algorithm learns to segment objects in static images from video training data. SANE uses background subtraction to find the segmentation of moving objects in videos. This provides object segmentation information for each video frame. The collection of frames and segmentations forms a training set that SANE uses to learn the image and shape properties of the observed motion boundaries. When presented with new static images, the trained model infers segmentations similar to the observed motion segmentations. SANE is a general method for learning environment-specific segmentation models. Because it can automatically generate training data from video, it can adapt to a new environment and new objects with relative ease, an advantage over untrained segmentation methods or those that require human-labeled training data. By using the local shape information in the training data, it outperforms a trained local boundary detector. Its performance is competitive with a trained top-down segmentation algorithm that uses global shape. The shape information it learns from one class of objects can assist the segmentation of other classes.
INDEX TERMS
Markov random fields, Computer vision
CITATION
Michael G. Ross, "Segmentation According to Natural Examples: Learning Static Segmentation from Motion Segmentation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 4, pp. 661-676, April 2009, doi:10.1109/TPAMI.2008.109
REFERENCES
[1] J. Besag, “Spatial Interaction and the Statistical Analysis of Lattice Systems,” J. Royal Statistical Soc., Series B (Methodological), vol. 36, no. 2, 1974.
[2] J. Besag, “On the Statistical Analysis of Dirty Pictures,” J. Royal Statistical Soc., Series B (Methodological), vol. 48, no. 3, 1986.
[3] E. Borenstein and S. Ullman, “Class-Specific, Top-Down Segmentation,” Proc. Seventh European Conf. Computer Vision, 2002.
[4] E. Borenstein and S. Ullman, “Learning to Segment,” Proc. Eighth European Conf. Computer Vision, 2004.
[5] J. Canny, “A Computational Approach to Edge Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, Nov. 1986.
[6] T. Cour, S. Yu, and J. Shi, Matlab Normalized Cuts Segmentation Code, http://www.cis.upenn.edu/~jshisoftware/, 2004.
[7] P. Fitzpatrick, “From First Contact to Close Encounters: A Developmentally Deep Perceptual System for a Humanoid Robot,” PhD dissertation, Massachusetts Inst. of Tech nology, 2003.
[8] W.T. Freeman, E.C. Pasztor, and O.T. Carmichael, “Learning Low-Level Vision,” Int'l J. Computer Vision, vol. 40, no. 1, 2000.
[9] A. Gelman, J.B. Carlin, H.S. Stern, and D.B. Rubin, Bayesian Data Analysis. Chapman & Hall/CRC, 2003.
[10] S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 6, Nov. 1984.
[11] T. Heskes, “On the Uniqueness of Loopy Belief Propagation Fixed Points,” Neural Computation, vol. 16, 2004.
[12] R. Jiroušek and S. Přeuoˇil, “On the Effective Implementation of the Iterative Proportional Fitting Procedure,” Computational Statistics & Data Analysis, vol. 19, pp. 177-189, 1995.
[13] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active Contour Models,” Proc. Int'l Conf. Computer Vision, 1987.
[14] S.M. Konishi, A.L. Yuille, J.M. Coughlan, and S.C. Zhu, “Statistical Edge Detection: Learning and Evaluating Edge Cues,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 1, Jan. 2003.
[15] M. Kumar, P. Torr, and A. Zisserman, “Obj Cut,” Proc. Computer Vision and Pattern Recognition, 2005.
[16] S. Kumar and M. Hebert, “Discriminative Random Fields: A Discriminative Framework for Contextual Interaction in Classification,” Proc. Int'l Conf. Computer Vision, 2003.
[17] J. Lafferty, A. McCallum, and F. Pereira, “Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data,” Proc. 18th Int'l Conf. Machine Learning, 2001.
[18] B. Leibe, A. Leonardis, and B. Schiele, “Combined Object Categorization and Segmentation with an Implicit Shape Model,” Proc. European Conf. Computer Vision Workshop Statistical Learning in Computer Vision, 2004.
[19] A. Levin and Y. Weiss, “Learning to Combine Top-Down and Bottom-Up Segmentation,” Proc. Ninth European Conf. Computer Vision, 2006.
[20] D. Marr, Vision. W.H. Freeman, 1982.
[21] D. Martin and C. Fowlkes, Matlab Boundary Detection Code, http://www.cs.berkeley.edu/projects/vision/ groupingsegbench/, 2004.
[22] D. Martin, C. Fowlkes, and J. Malik, “Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 5, May 2004.
[23] D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics,” Proc. Int'l Conf. Computer Vision, 2001.
[24] J. Migdal and W.E.L. Grimson, “Background Subtraction Using Markov Thresholds,” Proc. IEEE Workshop Motion and Video Computing, 2005.
[25] K.P. Murphy, Y. Weiss, and M.I. Jordan, “Loopy Belief Propagation for Approximate Inference: An Empirical Study,” Proc. 15th Conf. Uncertainty in Artificial Intelligence, 1999.
[26] S.E. Palmer, Vision Science: Photons to Phenomenology. The MIT Press, 1999.
[27] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
[28] X. Ren, C. Fowlkes, and J. Malik, “Scale-Invariant Contour Completion Using Conditional Random Fields,” Proc. Int'l Conf. Computer Vision, 2005.
[29] M.G. Ross, “Learning Static Object Segmentation from Motion Segmentation,” PhD dissertation, Massachusetts Inst. of Tech nology, 2005.
[30] M.G. Ross and L.P. Kaelbling, “Learning Object Segmentation from Video Data,” Technical Report AIM-2003-022, MIT Artificial Intelligence Laboratory, 2003.
[31] M.G. Ross and L.P. Kaelbling, “A Systematic Approach to Learning Object Segmentation from Motion,” Proc. NIPS Workshop Open Challenges in Cognitive Vision, 2003.
[32] M.G. Ross and L.P. Kaelbling, “Learning Static Object Segmentation from Motion Segmentation,” Proc. 20th Nat'l Conf. Artificial Intelligence, 2005.
[33] S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach. Prentice Hall, 2003.
[34] A. Shashua and S. Ullman, “Structural Saliency: The Detection of Globally Salient Structures Using a Locally Connected Network,” Proc. Int'l Conf. Computer Vision, 1988.
[35] J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” Proc. Computer Vision and Pattern Recognition, June 1997.
[36] E.S. Spelke, P. Vishton, and C. von Hofsten, “Object Perception, Object-Directed Action, and Physical Knowledge in Infancy,” The Cognitive Neurosciences, pp. 165-179, The MIT Press, 1994.
[37] C. Stauffer and W.E.L. Grimson, “Adaptive Background Mixture Models for Real-Time Tracking,” Proc. Computer Vision and Pattern Recognition, 1999.
[38] Z. Tu, X. Chen, A.L. Yuille, and S. Zhu, “Image Parsing: Unifying Segmentation, Detection, and Recognition,” Proc. Int'l Conf. Computer Vision, 2003.
[39] Z. Tu and S. Zhu, “Image Segmentation by Data-Driven Markov Chain Monte Carlo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, May 2002.
[40] M.J. Wainwright, T.S. Jaakkola, and A.S. Willsky, “Tree-Based Reparameterization Framework for Analysis of Sum-Product and Related Algorithms,” IEEE Trans. Information Theory, vol. 49, no. 5, May 2003.
[41] M.J. Wainwright, T.S. Jaakkola, and A.S. Willsky, “Tree-Reweighted Belief Propagation and Approximate ML Estimation by Pseudo-Moment Matching,” Proc. Ninth Int'l Workshop Artificial Intelligence and Statistics, 2003.
[42] M.J. Wainwright and E.B. Sudderth, Personal Communication, 2002.
[43] Y. Weiss, “Belief Propagation and Revision in Networks with Loops,” Technical Report 1616, MIT Artificial Intelligence Laboratory, Nov. 1997.
[44] Y. Weiss and W.T. Freeman, “On the Optimality of Solutions of the Max-Product Belief Propagation Algorithm in Arbitrary Graphs,” IEEE Trans. Information Theory, vol. 47, no. 2, 2001.
[45] J. Winn and N. Jojic, “Locus: Learning Object Classes with Unsupervised Segmentation,” Proc. Int'l Conf. Computer Vision, 2005.
[46] C. Yanover and Y. Weiss, “Finding the M Most Probable Configurations Using Loopy Belief Propagation,” Proc. Advances in Neural Information Processing Systems, 2003.
24 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool