The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - Feb. (2013 vol.35)
pp: 314-328
Yuewei Lin , Dept. of Comput. Sci. & Eng., Univ. of South Carolina, Columbia, SC, USA
Yuan Yan Tang , Dept. of Comput. & Inf. Sci., Univ. of Macau, Macau, China
Bin Fang , Coll. of Comput. Sci., Chongqing Univ., Chongqing, China
Zhaowei Shang , Coll. of Comput. Sci., Chongqing Univ., Chongqing, China
Yonghui Huang , Coll. of Comput. Sci., Chongqing Univ., Chongqing, China
Song Wang , Dept. of Comput. Sci. & Eng., Univ. of South Carolina, Columbia, SC, USA
ABSTRACT
This paper introduces a new computational visual-attention model for static and dynamic saliency maps. First, we use the Earth Mover's Distance (EMD) to measure the center-surround difference in the receptive field, instead of using the Difference-of-Gaussian filter that is widely used in many previous visual-attention models. Second, we propose to take two steps of biologically inspired nonlinear operations for combining different features: combining subsets of basic features into a set of super features using the Lm-norm and then combining the super features using the Winner-Take-All mechanism. Third, we extend the proposed model to construct dynamic saliency maps from videos by using EMD for computing the center-surround difference in the spatiotemporal receptive field. We evaluate the performance of the proposed model on both static image data and video data. Comparison results show that the proposed model outperforms several existing models under a unified evaluation setting.
INDEX TERMS
Computational modeling, Visualization, Histograms, Biological system modeling, Educational institutions, Humans, Earth,spatiotemporal receptive field (STRF), Visual attention, saliency maps, dynamic saliency maps, earth mover's distance (EMD)
CITATION
Yuewei Lin, Yuan Yan Tang, Bin Fang, Zhaowei Shang, Yonghui Huang, Song Wang, "A Visual-Attention Model Using Earth Mover's Distance-Based Saliency Measurement and Nonlinear Feature Combination", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 2, pp. 314-328, Feb. 2013, doi:10.1109/TPAMI.2012.119
REFERENCES
[1] E.A. Allen and R.D. Freeman, "Dynamic Spatial Processing Originates in Early Visual Pathways," J. Neuroscience, vol. 26, no. 45, pp. 11763-11774, 2006.
[2] T. Avraham and M. Lindenbaum, "Esaliency (Extended Saliency): Meaningful Attention Using Stochastic Image Modeling," IEEE Trans. Pattern Analysis Machine and Intelligence, vol. 32, no. 4, pp. 693-708, Apr. 2010.
[3] M.Z. Aziz and B. Mertsching, "Fast and Robust Generation of Feature Maps for Region-Based Visual Attention," IEEE Trans. Image Processing, vol. 17, no. 5, pp. 633-644, May 2008.
[4] N.D.B. Bruce and J.K. Tsotsos, "Saliency Based on Information Maximization," Proc. Advances in Neural Information Processing Systems, pp. 155-162, 2006.
[5] N.D.B. Bruce and J.K. Tsotsos, "Saliency, Attention, and Visual Search: An Information Theoretic Approach," J. Vision, vol. 9, no. 3, pp. 1-24, 2009.
[6] D. Cai, G.C. Deangelis, and R.D. Freeman, "Spatiotemporal Receptive Field Organization in the Lateral Geniculate Nucleus of Cats and Kittens," J. Neurophysiology, vol. 78, pp. 1045-1061, 1997.
[7] M. Corbetta and G.L. Shulman, "Control of Goal-Directed and Stimulus-Driven Attention in the Brain," Nature Rev. Neuroscience, vol. 3, no. 3, pp. 201-215, 2002.
[8] G.C. DeAngelis, I. Ohzawa, and R.D. Freeman, "Receptive-Field Dynamics in the Central Visual Pathways," Trends in Neurosciences, vol. 18, no. 10, pp. 451-458, 1995.
[9] B.A. Draper and A. Lionelle, "Evaluation of Selective Attention under Similarity Transformations," Computer Vision and Image Understanding, vol. 100, nos. 1/2, pp. 152-171, 2005.
[10] S. Frintrop, E. Rome, and H.I. Christensen, "Computational Visual Attention Systems and Their Cognitive Foundations: A Survey," ACM Trans. Applied Perception, vol. 7, no. 1,article 6, 2010.
[11] D. Gao and N. Vasconcelos, "Bottom-Up Saliency Is a Discriminant Process," Proc. IEEE Int'l Conf. Computer Vision, pp. 185-190, 2007.
[12] J. Harel, C. Koch, and P. Perona, "Graph-Based Visual Saliency," Proc. Neural Information Processing Systems, 2006.
[13] X. Hou and L. Zhang, "Saliency Detection: A Spectral Residual Approach," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[14] X. Hou and L. Zhang, "Dynamic Visual Attention: Searching for Coding Length Increments," Proc. Advances in Neural Information Processing Systems, pp. 681-688, 2008.
[15] D.H. Hubel and T.N. Wiesel, "Receptive-Field Dynamics in the Central Visual Pathways," J. Physiology, vol. 160, no. 1, pp. 106-154, 1962.
[16] L. Itti, C. Koch, and E. Niebur, "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis," IEEE Trans. Pattern Analysis Machine and Intelligence, vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
[17] L. Itti and C. Koch, "A Saliency Based Search Mechanism for Overt and Covert Shifts of Visual Attention," Vision Research, vol. 40, nos. 10-12, pp. 1489-1506, 2000.
[18] L. Itti and C. Koch, "Computational Modeling of Visual Attention," Nature Rev. Neuroscience, vol. 2, no. 3, pp. 194-203, 2001.
[19] L. Itti, "Automatic Foveation for Video Compression Using a Neurobiological Model of Visual Attention," IEEE Trans. Image Processing, vol. 13, no. 10, pp. 1304-1318, Oct. 2004.
[20] L. Itti and P. Baldi, "Bayesian Surprise Attracts Human Attention," Proc. Advances in Neural Information Processing Systems, vol. 18, pp. 547-554, 2006.
[21] L. Itti and P. Baldi, "Bayesian Surprise Attracts Human Attention," Vision Research, vol. 49, no. 10, pp. 1295-1306, 2009.
[22] T. Judd, K. Ehinger, F. Durand, and A. Torralba, "Learning to Predict Where Humans Look," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[23] E.I. Knudsen, "Fundamental Components of Attention," Ann. Rev. of Neuroscience, vol. 30, pp. 57-78, 2007.
[24] A.R. Koene and Z. Li, "Feature-Specific Interactions in Salience from Combined Feature Contrasts: Evidence for a Bottom-Up Saliency Map in V1," J. Vision, vol. 7, no. 7, pp. 1-14, 2007.
[25] E. Levina and P. Bickel, "The Earth Mover's Distance Is the Mallows Distance: Some Insights from Statistics," Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 251-256, 2001.
[26] Z. Li, "A Saliency Map in Primary Visual Cortex," Trends in Cognition Science, vol. 6, no. 1, pp. 9-16, 2002.
[27] Y. Lin, B. Fang, and Y.Y. Tang, "A Computational Model for Saliency Maps by Using Local Entropy," Proc. 24th AAAI Conf. Artificial Intelligence, pp. 967-973, 2010.
[28] H. Ling and K. Okada, "An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 5, pp. 840-853, May 2007.
[29] T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, and H.Y. Shum, "Learning to Detect a Salient Object," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 2, pp. 353-367, Feb. 2011.
[30] S.P. Liversedge and J.M. Findlay, "Saccadic Eye Movements and Cognition," Trends in Cognition Science, vol. 4, no. 1, pp. 6-14, 2000.
[31] D. Parkhurst, K. Law, and E. Niebur, "Modeling the Role of Salience in the Allocation of Overt Visual Attention," Vision Research, vol. 42, pp. 107-123, 2002.
[32] F. Poirier, F. Gosselin, and M. Arguin, "Perceptive Fields of Saliency," J. Vision, vol. 8, no. 15, pp. 1-19, 2008.
[33] U. Rajashekar, I. van der Linde, A.C. Bovik, and L.K. Cormack, "GAFFE: A Gaze-Attentive Fixation Finding Engine," IEEE Trans. Image Processing, vol. 17, no. 4, pp. 564-573, Apr. 2008.
[34] P. Reinagel and A.M. Zador, "Natural Scene Statistics at the Centre of Gaze," Network, vol. 10, pp. 341-350, 1999.
[35] M. Riesenhuber and T. Poggio, "Hierarchical Models of Object Recognition in Cortex," Nature Neuroscience, vol. 2, no. 11, pp. 1019-1025, Nov. 1999.
[36] Y. Rubner, C. Tomasi, and L.J. Guibas, "A Metric for Distributions with Applications to Image Databases," Proc. IEEE Int'l Conf. Computer Vision, pp. 59-66, Jan. 1998.
[37] Y. Rubner, C. Tomasi, and L.J. Guibas, "The Earth Mover's Distance as a Metric for Image Retrieval," Int'l J. Computer Vision, vol. 40, no. 2, pp. 99-121, 2000.
[38] F. Shic and B. Scassellati, "A Behavioral Analysis of Computational Models of Visual Attention," Int'l J. Computer Vision, vol. 73, no. 2, pp. 159-177, 2007.
[39] R.N. Shepard, "Attention and the Metric Structure of Stimulus Space," J. Math. Psychology, vol. 1, pp. 54-87, 1964.
[40] B. Tatler, R. Baddeley, and I. Gilchrist, "Visual Correlates of Fixation Selection: Effects of Scale and Time," Vision Research, vol. 45, no. 5, pp. 643-659, 2005.
[41] B.W. Tatler, "The Central Fixation Bias in Scene Viewing: Selecting an Optimal Viewing Position Independently of Motor Biases and Image Feature Distributions," J. Vision, vol. 7, no. 4, pp. 1-17, 2007.
[42] M. To, P.G. Lovell, T. Troscianko, and D.J. Tolhurst, "Summation of Perceptual Cues in Natural Visual Scenes," Proc. Royal Soc. B, vol. 275, no. 1649, pp. 2299-2308, 2008.
[43] A. Torralba, A. Oliva, M.S. Castelhano, and J.M. Henderson, "Contextual Guidance of Eye Movements and Attention in Real-World Scenes: The Role of Global Features on Object Search," Psychology Rev., vol. 113, no. 4, pp. 766-786, Oct. 2006.
[44] S. Treue, "Visual Attention: The Where, What, How and Why of Saliency," Current Opinion Neurobiology, vol. 13, no. 4, pp. 428-432, Aug. 2003.
[45] J.K. Tsotsos, "Analyzing Vision at the Complexity Level," Behavioral and Brain Sciences vol. 13, no. 3, pp. 423-445, 1990.
[46] D. Walther and C. Koch, "Modeling Attention to Salient Proto-Objects," Neural Networks, vol. 19, no. 9, pp. 1395-1407, 2006.
[47] J.M. Wolfe and T.S. Horowitz, "What Attributes Guide the Deployment of Visual Attention and How Do They Do It?" Nature Rev. Neuroscience, vol. 5, no. 6, pp. 495-501, June 2004.
[48] L. Zhang, M.H. Tong, T.K. Marks, H. Shan, and G.W. Cottrell, "SUN: A Bayesian Framework for Saliency Using Natural Statistics," J. Vision, vol. 8, no. 7, pp. 1-20, 2008.
68 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool