The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - Oct. (2012 vol.34)
pp: 1915-1926
Stas Goferman , Israel Institute of Technology, Haifa
Lihi Zelnik-Manor , Israel Institute of Technology, Haifa
Ayellet Tal , Israel Institute of Technology, Haifa
ABSTRACT
We propose a new type of saliency—context-aware saliency—which aims at detecting the image regions that represent the scene. This definition differs from previous definitions whose goal is to either identify fixation points or detect the dominant object. In accordance with our saliency definition, we present a detection algorithm which is based on four principles observed in the psychological literature. The benefits of the proposed approach are evaluated in two applications where the context of the dominant objects is just as essential as the objects themselves. In image retargeting, we demonstrate that using our saliency prevents distortions in the important regions. In summarization, we show that our saliency helps to produce compact, appealing, and informative summaries.
INDEX TERMS
Feature extraction, Context awareness, Visualization, Object recognition, Estimation, Image color analysis, Human factors, context aware., Image saliency, visual saliency
CITATION
Stas Goferman, Lihi Zelnik-Manor, Ayellet Tal, "Context-Aware Saliency Detection", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 10, pp. 1915-1926, Oct. 2012, doi:10.1109/TPAMI.2011.272
REFERENCES
[1] R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, "Frequency-Tuned Salient Region Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1597-1604, 2009.
[2] ANN, http://www.cs.umd.edu/mountANN/, 2011.
[3] S. Arya, D. Mount, N. Netanyahu, R. Silverman, and A. Wu, "An Optimal Algorithm for Approximate Nearest Neighbor Searching Fixed Dimensions," J. ACM, vol. 45, no. 6, pp. 891-923, 1998.
[4] S. Avidan and A. Shamir, "Seam Carving for Content-Aware Image Resizing," ACM Trans. Graphics, vol. 26, no. 3, p. 10, 2007.
[5] O. Boiman and M. Irani, "Detecting Irregularities in Images and in Video," Int'l J. Computer Vision, vol. 74, no. 1, pp. 17-31, 2007.
[6] N. Bruce and J. Tsotsos, "Saliency Based on Information Maximization," Advances in Neural Information Processing Systems, vol. 18, pp. 155-162, 2006.
[7] V. Garcia, "Suivi Dobjets Dintrt dans une Squence Dimages: Des Points Saillants aux Mesures Statistiques," PhD thesis, Universit de Nice, 2008.
[8] V. Garcia, E. Debreuve, and M. Barlaud, "Fast k Nearest Neighbor Search Using GPU," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[9] S. Goferman, A. Tal, and L. Zelnik-Manor, "Puzzle-Like Collage," Computer Graphics Forum, vol. 29, pp. 459-468, 2010.
[10] S. Goferman, L. Zelnik-Manor, and A. Tal, "Context-Aware Saliency Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2376-2383, 2010.
[11] GPU, http://www.i3s.unice.fr/~creativeKNN/, 2011.
[12] C. Guo, Q. Ma, and L. Zhang, "Spatio-Temporal Saliency Detection Using Phase Spectrum of Quaternion Fourier Transform," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[13] J. Harel, C. Koch, and P. Perona, "Graph-Based Visual Saliency," Advances in Neural Information Processing Systems, vol. 19, pp. 545-552, 2007.
[14] X. Hou and L. Zhang, "Saliency Detection: A Spectral Residual Approach," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[15] L. Itti and C. Koch, "Computational Modelling of Visual Attention," Nature Rev. Neuroscience, vol. 2, no. 3, pp. 194-204, 2001.
[16] L. Itti, C. Koch, and E. Niebur, "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
[17] T. Judd, K. Ehinger, F. Durand, and A. Torralba, "Learning to Predict Where Humans Look," Proc. IEEE Int'l Conf. Computer Vision, pp. 2106-2113, 2009.
[18] C. Koch and T. Poggio, "Predicting the Visual World: Silence Is Golden," Nature Neuroscience, vol. 2, pp. 9-10, 1999.
[19] K. Koffka, Principles of Gestalt Psychology. Routledge & Kegan Paul, 1955.
[20] O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, "A Coherent Computational Approach to Model Bottom-up Visual Attention," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 5, pp. 802-817, May 2006.
[21] T. Liu, J. Sun, N. Zheng, X. Tang, and H. Shum, "Learning to Detect a Salient Object," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[22] Y. Ma and H. Zhang, "Contrast-Based Image Attention Analysis by Using Fuzzy Growing," Proc. 11th ACM Int'l Conf. Multimedia, pp. 374-381, 2003.
[23] E. Nowak, F. Jurie, and B. Triggs, "Sampling Strategies for Bag-of-Features Image Classification," Proc. Ninth European Conf. Computer Vision, pp. 490-503, 2006.
[24] Y. Pritch, E. Kav-Venaki, and S. Peleg, "Shift Map Image Editing," Proc. 12th IEEE Int'l Conf. Computer Vision, pp. 151-158, 2009.
[25] Z.W.M.C.Q. Lv, W. Josephson, and K. Li, "Multiprobe LSH: Efficient Indexing for High-Dimensional Similarity Search," Proc. 33rd Int'l Conf. Very Large Data Bases, pp. 950-961, 2007.
[26] E. Rahtu, J. Kannala, M. Salo, and J. Heikkilä, "Segmenting Salient Objects from Images and Videos," Proc. 11th European Conf. Computer Vision, pp. 366-379, 2010.
[27] C. Rother, L. Bordeaux, Y. Hamadi, and A. Blake, "Autocollage," ACM Trans. Graphics, vol. 25, no. 3, pp. 847-852, 2006.
[28] C. Rother, V. Kolmogorov, and A. Blake, "'GrabCut': Interactive Foreground Extraction Using Iterated Graph Cuts," ACM Trans. Graphics, vol. 23, no. 3, pp. 309-314, 2004.
[29] M. Rubinstein, A. Shamir, and S. Avidan, "Improved Seam Carving for Video Retargeting," ACM Trans. Graphics, vol. 27, no. 3, p. 16, 2008.
[30] U. Rutishauser, D. Walther, C. Koch, and P. Perona, "Is Bottom-Up Attention Useful for Object Recognition?," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, 2004.
[31] H. Seo and P. Milanfar, "Static and Space-Time Visual Saliency Detection by Self-Resemblance," J. Vision, vol. 9, no. 12, pp. 1-27, 2009.
[32] B. Suh, H. Ling, B.B. Bederson, and D.W. Jacobs, "Automatic Thumbnail Cropping and Its Effectiveness," Proc. 16th Ann. ACM Symp. User Interface Software and Technology, pp. 95-104, 2003.
[33] A. Treisman and G. Gelade, "A Feature-Integration Theory of Attention," Cognitive Psychology, vol. 12, no. 1, pp. 97-136, 1980.
[34] P. Viola and M. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features," Proc. IEEE Computer Vision and Pattern Recognition, 2001.
[35] D. Walther and C. Koch, "Modeling Attention to Salient Proto-Objects," Neural Networks, vol. 19, no. 9, pp. 1395-1407, 2006.
[36] J. Wolfe, "Guided Search 2.0: A Revised Model of Visual Search," Psychonomic Bull. and Rev., vol. 1, no. 2, pp. 202-238, 1994.
51 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool