This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Active Visual Segmentation
April 2012 (vol. 34 no. 4)
pp. 639-653
ASCII Text
x 
 
A. K. Mishra, Y. Aloimonos, Loong Fah Cheong, A. Kassim, "Active Visual Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 4, pp. 639-653, April, 2012.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2011.171,
author = {A. K. Mishra and Y. Aloimonos and Loong Fah Cheong and A. Kassim},
title = {Active Visual Segmentation},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {34},
number = {4},
issn = {0162-8828},
year = {2012},
pages = {639-653},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2011.171},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Active Visual Segmentation
IS - 4
SN - 0162-8828
SP639
EP653
EPD - 639-653
A1 - A. K. Mishra,
A1 - Y. Aloimonos,
A1 - Loong Fah Cheong,
A1 - A. Kassim,
PY - 2012
KW - probability
KW - computer vision
KW - image segmentation
KW - feedback process
KW - active visual segmentation
KW - human visual system
KW - visual attention
KW - fixation point
KW - Cartesian space
KW - probabilistic boundary edge map
KW - mid-level visual cue
KW - low-level visual cue
KW - segmentation refinement process
KW - Image edge detection
KW - Image segmentation
KW - Probabilistic logic
KW - Visual system
KW - Humans
KW - Visualization
KW - Image color analysis
KW - visual attention.
KW - Fixation-based segmentation
KW - object segmentation
KW - polar space
KW - cue integration
KW - scale invariance
VL - 34
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
A. K. Mishra, Dept. of Comput. Sci., Univ. of Maryland, College Park, MD, USA
Y. Aloimonos, Dept. of Comput. Sci., Univ. of Maryland, College Park, MD, USA
Loong Fah Cheong, Electr. & Comput. Eng., Nat. Univ. of Singapore, Singapore, Singapore
A. Kassim, Electr. & Comput. Eng., Nat. Univ. of Singapore, Singapore, Singapore
Attention is an integral part of the human visual system and has been widely studied in the visual attention literature. The human eyes fixate at important locations in the scene, and every fixation point lies inside a particular region of arbitrary shape and size, which can either be an entire object or a part of it. Using that fixation point as an identification marker on the object, we propose a method to segment the object of interest by finding the “optimal” closed contour around the fixation point in the polar space, avoiding the perennial problem of scale in the Cartesian space. The proposed segmentation process is carried out in two separate steps: First, all visual cues are combined to generate the probabilistic boundary edge map of the scene; second, in this edge map, the “optimal” closed contour around a given fixation point is found. Having two separate steps also makes it possible to establish a simple feedback between the mid-level cue (regions) and the low-level visual cues (edges). In fact, we propose a segmentation refinement process based on such a feedback process. Finally, our experiments show the promise of the proposed method as an automatic segmentation framework for a general purpose visual system.

[1] R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, "Frequency-Tuned Salient Region Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[2] S. Alpert, M. Galun, R. Basri, and A. Brandt, "Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2007.
[3] P. Arbelaez and L. Cohen, "Constrained Image Segmentation from Hierarchical Boundaries," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 454-467, 2008.
[4] S. Bagon, O. Boiman, and M. Irani, "What Is a Good Image Segment? A Unified Approach to Segment Extraction," Proc. 10th European Conf. Computer Vision, pp. 30-44, 2008.
[5] A. Blake, C. Rother, M. Brown, P. Perez, and P. Torr, "Interactive Image Segmentation Using an Adaptive GMMRF Model," Proc. European Conf. Computer Vision, pp. 428-441, 2004.
[6] Y.Y. Boykov and M.P. Jolly, "Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in n-d Images," Proc. Eighth IEEE Int'l Conf. Computer Vision, pp. 105-112, 2001.
[7] Y.Y. Boykov and V. Kolmogorov, "An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1124-1137, Sept. 2004.
[8] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, High Accuracy Optical Flow Estimation Based on a Theory for Warping, pp. 25-36. Springer, 2004.
[9] N.D.B. Bruce and J.K. Tsotsos, "Saliency, Attention, and Visual Search: An Information Theoretic Approach," J. Vision, vol. 9, no. 3, pp. 1-24, 2009.
[10] M. Cerf, J. Harel, W. Einhäuser, and C. Koch, "Predicting Human Gaze Using Low-Level Saliency Combined with Face Detection," Proc. Neural Information Processing Systems, 2008.
[11] E. Craft, H. Schütze, E. Niebur, and R. von der Heydt, "A Neural Model of Figure-Ground Organization," J. Neurophysiology, vol. 6, no. 97, pp. 4310-4326, 2007.
[12] P. Dimitrov, C. Phillips, and K. Siddiqi, "Robust and Efficient Skeletal Graphs," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 417-423, 2000.
[13] L. Ding and M.A. Martinez, "Features versus Context: An Approach for Precise and Detailed Detection and Delineation of Faces and Facial Features," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 11, pp. 2022-2038, Nov. 2010.
[14] P.F. Felzenszwalb and D.P. Huttenlocher, "Efficient Graph-Based Image Segmentation," Int'l J. Computer Vision, vol. 59, no. 2, pp. 167-181, 2004.
[15] V. Ferrari, T. Tuytelaars, and L.V. Gool, "Object Detection by Contour Segment Networks," Proc. European Conf. Computer Vision, pp. 14-28, June 2006.
[16] M. Gur, A. Beylin, and D.M. Snodderly, "Response Variability of Neurons in Primary Visual Cortex (V1) of Alert Monkeys," J. Neuroscience, vol. 17, pp. 2914-2920, 1997.
[17] J.M. Henderson and A. Hollingworth, Eye Movements during Scene Viewing: An Overview. Oxford, 1998.
[18] J.M. Henderson, C.C. Williams, M.S. Castelhano, and R.J. Falk, "Eye Movements and Picture Processing during Recognition," Perception and Psychophysics, vol. 65, pp. 725-734, 2003.
[19] A. Hollingworth, G. Schrock, and J.M. Henderson, "Change Detection in the Flicker Paradigm: The Role of Fixation Position within the Scene," Memory and Cognition, vol. 29, pp. 296-304, 2001.
[20] L. Itti, C. Koch, and E. Niebur, "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
[21] V. Kolmogorov, A. Criminisi, A. Blake, G. Cross, and C. Rother, "Bi-Layer Segmentation of Binocular Stereo Video," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 407-414, 2005.
[22] E. Kowler and R.M. Steinman, "Small Saccades Serve No Useful Purpose: Reply to a Letter by R. W. Ditchburn," Vision Research, vol. 20, pp. 273-276, 1980.
[23] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision , vol. 60, no. 2, pp. 91-110, 2004.
[24] D. Martin, C. Fowlkes, and J. Malik, "Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 5, pp. 530-549, May 2004.
[25] D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics," Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 416-423, July 2001.
[26] A.M. Martínez, P. Mittrapiyanuruk, and A.C. Kak, "On Combining Graph-Partitioning with Non-Parametric Clustering for Image Segmentation," Computer Vision and Image Understanding, vol. 95, pp. 72-85, July 2004.
[27] S. Martinez-Conde, S.L. Macknik, and D.H. Hubel, "The Role of Fixational Eye Movements in Visual Perception," Nature Rev. Neuroscience, vol. 5, pp. 229-240, 2004.
[28] K. Mikolajczyk and C. Schmid, "An Affine Invariant Interest Point Detector," Proc. Seventh European Conf. Computer Vision, pp. 128-142, 2002.
[29] A.S. Ogale and Y. Aloimonos, "A Roadmap to the Integration of Early Visual Modules," Int'l J. Computer Vision, vol. 72, no. 1, pp. 9-25, Apr. 2007.
[30] D. Parkhurst, K. Law, and E. Niebur, "Modeling the Role of Salience in the Allocation of Overt Visual Attention," Vision Research, vol. 42, pp. 107-23, 2000.
[31] U. Rajashekar, I. van der Linde, A.C. Bovik, and L.K. Cormack, "Gaffe: A Gaze-Attentive Fixation Finding Engine," IEEE Trans. Image Processing, vol. 17, no. 4, pp. 564-573, Apr. 2008.
[32] A.L. Rothenstein and J.K. Tsotsos, "Attention Links Sensing to Recognition," Image Vision Computing, vol. 26, no. 1, pp. 114-126, 2008.
[33] C. Rother, V. Kolmogorov, and A. Blake, "'GrabCut': Interactive Foreground Extraction Using Iterated Graph Cuts," ACM Trans. Graphics, vol. 23, no. 3, pp. 309-314, 2004.
[34] J.T. Serences and S. Yantis, "Selective Visual Attention and Perceptual Coherence," Trends in Cognitive Sciences, vol. 10, no. 1, pp. 38-45, 2006.
[35] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[36] C. Siagian and L. Itti, "Rapid Biologically-Inspired Scene Classification Using Features Shared with Visual Attention," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 300-312, Feb. 2007.
[37] A.K. Sinop and L. Grady, "A Seeded Image Segmentation Framework Unifying Graph Cuts and Random Walker Which Yields a New Algorithm," Proc. IEEE 11th Int'l Conf. Computer Vision, pp. 1-8, 2007.
[38] R.H. Steinberg, M. Reid, and P.L. Lacy, "The Distribution of Rods and Cones in the Retina of the Cat (Fells Domestica)," J. Computational Neuroscience, vol. 148, pp. 229-248, 1973.
[39] X.Y. Stella and J. Shi, "Grouping with Bias," Proc. Neural Information Processing Systems, 2001.
[40] O.M. Thomas, B.G. Cumming, and A.J. Parker, "A Specialization for Relative Disparity in V2," Nature Neuroscience, vol. 5, no. 5, pp. 472-478, May 2002.
[41] A. Torralba, A. Oliva, M.S. Castelhano, and J.M. Henderson, "Contextual Guidance of Eye Movements and Attention in Real-World Scenes: The Role of Global Features in Object Search," Psychological Rev., vol. 113, no. 4, pp. 766-786, 2006.
[42] A. Toshev, A. Makadia, and K. Daniilidis, "Shape-Based Object Recognition in Videos Using 3D Synthetic Object Models," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[43] Z.W. Tu and S.C. Zhu, "Mean Shift: A Robust Approach Toward Feature Space Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[44] O. Veksler, "Star Shape Prior for Graph-Cut Image Segmentation," Proc. 10th European Conf. Computer Vision, vol. 3, pp. 454-467, 2008.
[45] D. Walther and C. Koch, "Modeling Attention to Salient Proto-Objects," Neural Networks, vol. 19, no. 4, pp. 1395-1407, Apr. 2006.
[46] K.C. Winkler, R.W. Williams, and P. Rakic, "Photoreceptor Mosaic: Number and Distribution of Rods and Cones in the Rhesus Monkey," J. Computational Neuroscience, vol. 297, pp. 499-508, 1990.

Index Terms:
probability,computer vision,image segmentation,feedback process,active visual segmentation,human visual system,visual attention,fixation point,Cartesian space,probabilistic boundary edge map,mid-level visual cue,low-level visual cue,segmentation refinement process,Image edge detection,Image segmentation,Probabilistic logic,Visual system,Humans,Visualization,Image color analysis,visual attention.,Fixation-based segmentation,object segmentation,polar space,cue integration,scale invariance
Citation:
A. K. Mishra, Y. Aloimonos, Loong Fah Cheong, A. Kassim, "Active Visual Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 4, pp. 639-653, April 2012, doi:10.1109/TPAMI.2011.171
Usage of this product signifies your acceptance of the Terms of Use.