The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - December (2008 vol.30)
pp: 2109-2125
ABSTRACT
We construct a segmentation scheme that combines top-down with bottom-up processing. In the proposed scheme, segmentation and recognition are intertwined rather than proceeding in a serial manner. The top-down part applies stored knowledge about object shapes acquired through learning, whereas the bottom-up part creates a hierarchy of segmented regions based on uniformity criteria. Beginning with unsegmented training examples of class and non-class images, the algorithm constructs a bank of class-specific fragments and determines their figure-ground segmentation. This bank is then used to segment novel images in a top-down manner: the fragments are first used to recognize images containing class objects, and then to create a complete cover that best approximates these objects. The resulting segmentation is then integrated with bottom-up multi-scale grouping to better delineate the object boundaries. Our experiments, applied to a large set of four classes (horses, pedestrians, cars, faces), demonstrate segmentation results that surpass those achieved by previous top-down or bottom-up schemes. The main novel aspects of this work are the fragment learning phase, which efficiently learns the figure-ground labeling of segmentation fragments, even in training sets with high object and background variability; combining the top-down segmentation with bottom-up criteria to draw on their relative merits; and the use of segmentation to improve recognition.
INDEX TERMS
Vision and Scene Understanding, Perceptual reasoning
CITATION
Eran Borenstein, Shimon Ullman, "Combined Top-Down/Bottom-Up Segmentation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 12, pp. 2109-2125, December 2008, doi:10.1109/TPAMI.2007.70840
REFERENCES
[1] E. Borenstein and S. Ullman, “Class-Specific, Top-Down Segmentation,” Proc. European Conf. Computer Vision (ECCV '02), vol. 2, pp.109-124, 2002.
[2] E. Borenstein and S. Ullman, “Learning to Segment,” Proc. European Conf. Computer Vision (ECCV), 2004.
[3] E. Borenstein, E. Sharon, and S. Ullman, “Combining Top-Down and Bottom-Up Segmentation,” Proc. Computer Vision and Pattern Recognition Workshop Perceptual Organization in Computer Vision, 2004.
[4] A. Weeks and G. Hague, “Color Segmentation in the HSI Color Space Using the k-Means Algorithm,” Proc. SPIE, vol. 3026, pp.143-154, Feb. 1997.
[5] J.D. Buf, M. Kardan, and M. Spann, “Texture Feature Performance for Image Segmentation,” Pattern Recognition, vol. 23, 1990.
[6] U. Montanari, “On the Optimal Detection of Curves in Noisy Pictures,” Comm. ACM, vol. 14, 1971.
[7] A. Shashua and S. Ullman, “Structural Saliency: The Detection of Globally Salient Structures Using a Locally Connected Network,” Proc. Int'l Conf. Computer Vision, 1988.
[8] S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distribution, and the Bayesian Restoration of Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, Nov. 1984.
[9] D. Mumford and J. Shah, “Boundary Detection by Minimizing Functionals,” Proc. Computer Vision and Pattern Recognition (CVPR '85), 1985.
[10] J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” Proc. Computer Vision and Pattern Recognition (CVPR), 1997.
[11] A. Yuille and P. Hallinan, “Deformable Templates,” Active Vision, A. Blake and A. Yuille, eds., pp. 21-38, MIT Press, 1992.
[12] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active Contour Models,” Int'l J. Computer Vision, vol. 1, pp. 321-331, 1987.
[13] V. Caselles, F. Catte, T. Coll, and F. Dibos, “A Geometric Model for Active Contours in Image Processing,” Numerische Mathematik, vol. 66, pp. 1-31, 1993.
[14] R. Malladi, J. Sethian, and B. Vemuri, “Shape Modeling with Front Propagation: A Level Set Approach,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 2, pp. 158-175, 1995.
[15] V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic Active Contours,” IEEE Trans. Pattern Analysis and Machine Intelligence, 1995.
[16] B. Leibe and B. Schiele, “Interleaved Object Categorization and Segmentation,” Proc. British Machine Vision Conf. (BMVC), 2003.
[17] E. Bernstein and Y. Amit, “Part-Based Statistical Models for Object Classification and Detection,” Proc. Computer Vision and Pattern Recognition (CVPR '05), vol. 2, 2005.
[18] M.P. Kumar, P.H.S. Torr, and A. Zisserman, “Obj cut,” Proc. Computer Vision and Pattern Recognition (CVPR '05), vol. 1, pp. 18-25, 2005.
[19] J. Winn and N. Jojic, “Locus: Learning Object Classes with Unsupervised Segmentation,” Proc. Int'l Conf. Computer Vision (ICCV '05), vol. 1, pp. 756-763, 2005.
[20] J. Winn and J. Shotton, “The Layout Consistent Random Field for Recognizing and Segmenting Partially Occluded Objects,” Proc. Computer Vision and Pattern Recognition (CVPR), 2006.
[21] S.X. Yu, R. Gross, and J. Shi, “Concurrent Object Recognition and Segmentation by Graph Partitioning,” Proc. Ann. Conf. Advances in Neural Information Processing Systems (NIPS), 2002.
[22] X. Chen, Z. Tu, A. Yuille, and S. Zhu, “Image Parsing: Segmentation, Detection and Recognition,” Proc. Int'l Conf. Computer Vision (ICCV), 2003.
[23] L. Liu and S. Sclaroff, “Region Segmentation via Deformable Model-Guided Split and Merge,” Proc. Int'l Conf. Computer Vision (ICCV), 2001.
[24] G. Mori, X. Ren, A. Efros, and J. Malik, “Recovering Human Body Configurations: Combining Segmentation and Recognition,” Proc. Computer Vision and Pattern Recognition (CVPR), 2004.
[25] X. Ren, C. Fowlkes, and J. Malik, “Scale-Invariant Contour Completion Using Conditional Random Fields,” Proc. Int'l Conf. Computer Vision (ICCV), 2005.
[26] L. Zhao and L.S. Davis, “Closely Coupled Object Detection and Segmentation,” Proc. Int'l Conf. Computer Vision (ICCV '05), vol. 1, 2005.
[27] A. Levin and Y. Weiss, “Learning to Combine Bottom-Up and Top-Down Segmentation,” Proc. European Conf. Computer Vision (ECCV), 2006.
[28] M. Peterson, “Object Recognition Processes Can and Do Operate Before Figure-Ground Organization,” Current Directions in Psychological Science, vol. 3, pp. 105-111, 1994.
[29] M.J. Brady and D. Kersten, “Bootstrapped Learning of Novel Objects,” J. Vision, vol. 3, no. 6, pp. 413-422, 2003.
[30] A. Needham, “Object Recognition and Object Segregation in 4.5-Month-Old Infants,” J. Experimental Child Psychology, vol. 78, pp. 3-24, 2001.
[31] P.C. Quinn and P.G. Schyns, “What Goes Up May Come Down: Perceptual Process and Knowledge Access in the Organization of Complex Visual Patterns by Young Infants,” Congnitive Science, vol. 27, no. 6, pp. 923-935, 2003.
[32] R.S. Zemel, M. Behrmann, M.C. Mozer, and D. Bavelier, “Object Recognition Processes Can and Do Operate Before Figure-Ground Organization,” Experimental Psychology, vol. 28, no. 1, pp. 202-217, Feb. 2002.
[33] K. Zipser, V. Lamme, and P.H. Schiller, “Contextual Modulation in Primary Visual Cortex,” J. Neuroscience, vol. 16, no. 22, pp. 7376-7389, 1996.
[34] V. Lamme, “The Neurophysiology of Figure-Ground Segregation in Primary Visual Cortex,” J. Neuroscience, vol. 15, no. 2, pp. 1605-1615, Feb. 1995.
[35] G.C. Baylis and J. Driver, “Shape-Coding in IT Cells Generalizes over Contrast and Mirror Reversal, but Not Figure-Ground Reversal,” Nature Neuroscience, vol. 4, no. 9, pp. 937-942, 2001.
[36] J. Hupe, A. James, B. Payne, S. Lomber, and J. Bullier, “Cortical Feedback Improves Discrimination between Figure and Background by v1, v2 and v3 Neurons,” Nature, vol. 394, pp. 784-787, Aug. 1998.
[37] H. Supper, H. Spekreijse, and V. Lamme, “Contextual Modulation in Primary Visual Cortex as a Neuronal Substrate for Working Memory,” J. Vision, vol. 1, no. 3, p. 345, 2001.
[38] M.C. Burl, M. Weber, and P. Perona, “A Probabilistic Approach to Object Recognition Using Local Photometry and Global Geometry,” LNCS 1407, 1998.
[39] S. Agarwal and D. Roth, “Learning a Sparse Representation for Object Detection,” Proc. European Conf. Computer Vision (ECCV '02), vol. 4, pp. 113-130, 2002.
[40] S. Ullman and E. Sali, “Object Classification Using a Fragment-Based Representation,” Proc. First IEEE Int'l Workshop Biologically Motivated Computer Vision (BMCV '00), pp. 73-87, 2000.
[41] F. Fleuret, “Fast Binary Feature Selection with Conditional Mutual Information,” J. Machine Learning Research, Nov. 2004.
[42] D.N. Bhat and S.K. Nayar, “Ordinal Measures for Image Correspondence,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 415-423, Apr. 1998.
[43] D. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[44] K. Mikolajczyk and C. Schmid, “An Affine Invariant Interest Point Detector,” Proc. European Conf. Computer Vision (ECCV '02), vol. 1, p. 128, 2002.
[45] S. Ullman, E. Sali, and M. Vidal-Naquet, “A Fragment Based Approach to Object Representation and Classification,” Proc. Fourth Int'l Workshop Visual Form, 2001.
[46] M. Galun, E. Sharon, R. Basri, and A. Brandt, “Texture Segmentation by Multiscale Aggregation of Filter Responses and Shape Elements,” Proc. Int'l Conf. Computer Vision (ICCV '03), pp.716-723, 2003.
[47] F. Kschischang, B. Frey, and H. Loeliger, “Factor Graphs and the Sum-Product Algorithm,” IEEE Trans. Information Theory, vol. 47, pp. 498-519, Feb. 2001.
[48] E. Sali and S. Ullman, “Detecting Object Classes by the Detection of Overlapping 2-D Fragments,” Proc. British Machine Vision Conf. (BMVC), 1999.
[49] Y. Amit, “A Neural Network Architecture for Visual Selection,” Neural Computation, vol. 12, no. 5, pp. 1141-1164, 2000.
[50] H. Schneiderman and T. Kanade, “Object Detection Using the Statistics of Parts,” Int'l J. Computer Vision, vol. 56, no. 3, pp. 151-177, 2004.
[51] E. Sharon, M. Galun, D. Sharon, R. Basri, and A. Brandt, “Hierarchy and Adaptivity in Segmenting Visual Scenes,” Nature, vol. 442, pp. 810-813, 2006.
[52] M. Oren, C. Papageorgiou, P. Sinha, E. Osuna, and T. Poggio, “Pedestrian Detection Using Wavelet Templates,” Proc. Computer Vision and Pattern Recognition (CVPR '97), pp. 193-199, 1997.
6 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool