The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan. (2013 vol.35)
pp: 185-207
A. Borji , Dept. of Comput. Sci., Univ. of Southern California, Los Angeles, CA, USA
L. Itti , Dept. of Comput. Sci., Univ. of Southern California, Los Angeles, CA, USA
ABSTRACT
Modeling visual attention-particularly stimulus-driven, saliency-based attention-has been a very active research area over the past 25 years. Many different models of attention are now available which, aside from lending theoretical contributions to other fields, have demonstrated successful applications in computer vision, mobile robotics, and cognitive systems. Here we review, from a computational perspective, the basic concepts of attention implemented in these models. We present a taxonomy of nearly 65 models, which provides a critical comparison of approaches, their capabilities, and shortcomings. In particular, 13 criteria derived from behavioral and computational studies are formulated for qualitative comparison of attention models. Furthermore, we address several challenging issues with models, including biological plausibility of the computations, correlation with eye movement datasets, bottom-up and top-down dissociation, and constructing meaningful performance measures. Finally, we highlight current research trends in attention modeling and provide insights for future.
INDEX TERMS
Computational modeling, Visualization, Hidden Markov models, Feature extraction, Humans, Solid modeling, Search problems,gist, Visual attention, bottom-up attention, top-down attention, saliency, eye movements, regions of interest, gaze control, scene interpretation, visual search
CITATION
A. Borji, L. Itti, "State-of-the-Art in Visual Attention Modeling", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 1, pp. 185-207, Jan. 2013, doi:10.1109/TPAMI.2012.89
REFERENCES
[1] K. Koch, J. McLean, R. Segev, M.A. Freed, M.J. Berry, V. Balasubramanian, and P. Sterling, "How Much the Eye Tells the Brain," Current Biology, vol. 25, nos. 16-14, pp. 1428-34, 2006.
[2] L. Itti, "Models of Bottom-Up and Top-Down Visual Attention," PhD thesis, California Inst. of Tech nology, 2000.
[3] D.J. Simons and D.T. Levin, "Failure to Detect Changes to Attended Objects," Investigative Ophthalmology and Visual Science, vol. 38, no. 4, p. 3273, 1997.
[4] R.A. Rensink, "How Much of a Scene Is Seen—The Role of Attention in Scene Perception," Investigative Ophthalmology and Visual Science, vol. 38, p. 707, 1997.
[5] D.J. Simons and C.F. Chabris, "Gorillas in Our Midst: Sustained Inattentional Blindness for Dynamic Events," Perception, vol. 28, no. 9, pp. 1059-1074, 1999.
[6] J.E. Raymond, K.L. Shapiro, and K.M. Arnell, "Temporary Suppression of Visual Processing in an RSVP Task: An Attentional Blink?" J. Experimental Psychology, vol. 18, no 3, pp. 849-60, 1992.
[7] S. Treue and J.H.R. Maunsell, "Attentional Modulation of Visual Motion Processing in Cortical Areas MT and MST," Nature, vol. 382, pp. 539-541, 1996.
[8] S. Frintrop, E. Rome, and H.I. Christensen, "Computational Visual Attention Systems and Their Cognitive Foundations: A Survey," ACM Trans. Applied Perception, vol. 7, no. 1, Article 6, 2010.
[9] A. Rothenstein and J. Tsotsos, "Attention Links Sensing to Recognition," J. Image and Vision Computing, vol. 26, pp. 114-126, 2006.
[10] R. Desimone and J. Duncan, "Neural Mechanisms of Selective Visual Attention," Ann. Rev. Neuroscience, vol. 18, pp. 193-222, 1995.
[11] S.J. Luck, L. Chelazzi, S.A. Hillyard, and R. Desimone, "Neural Mechanisms of Spatial Selective Attention in Areas V1, V2, and V4 of Macaque Visual Cortex," J. Neurophysiology, vol. 77, pp. 24-42, 1997.
[12] C. Bundesen and T. Habekost, "Attention," Handbook of Cognition, K. Lamberts and R. Goldstone, eds., 2005.
[13] V. Navalpakkam, C. Koch, A. Rangel, and P. Perona, "Optimal Reward Harvesting in Complex Perceptual Environments," Proc. Nat'l Academy of Sciences USA, vol. 107, no. 11, pp. 5232-5237, 2010.
[14] L. Itti, C. Koch, and E. Niebur, "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
[15] J.K. Tsotsos, S.M. Culhane, W.Y.K. Wai, Y. Lai, N. Davis, and F. Nuflo, "Modeling Visual Attention via Selective Tuning," Artificial Intelligence, vol. 78, nos. 1-2, pp. 507-545, 1995.
[16] R. Milanese, "Detecting Salient Regions in an Image: From Biological Evidence to Computer Implementation," PhD thesis, Univ. Geneva, 1993.
[17] S. Baluja and D. Pomerleau, "Using a Saliency Map for Active Spatial Selective Attention: Implementation & Initial Results," Proc. Advances in Neural Information Processing Systems, pp. 451-458, 1994.
[18] C. Koch and S. Ullman, "Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry," Human Neurobiology, vol. 4, no. 4, pp. 219-227, 1985.
[19] K. Rayner, "Eye Movements in Reading and Information Processing: 20 Years of Research," Psychological Bull., vol. 134, pp. 372-422, 1998.
[20] J. Najemnik and W.S. Geisler, "Optimal Eye Movement Strategies in Visual Search," Nature, vol. 434, pp. 387-391, 2005.
[21] L.W. Renninger, J.M. Coughlan, P. Verghese, and J. Malik, "An Information Maximization Model of Eye Movements," Advances in Neural Information Processing Systems, vol. 17, pp. 1121-1128, 2005.
[22] U. Rutishauser and C. Koch, "Probabilistic Modeling of Eye Movement Data during Conjunction Search via Feature-Based Attention," J. Vision, vol. 7, no. 6, pp. 1-20, 2007.
[23] R. Rao, G. Zelinsky, M. Hayhoe, and D. Ballard, "Eye Movements in Iconic Visual Search," Vision Research, vol. 42, pp. 1447-1463, 2002.
[24] A.T. Duchowski, "A Breadth-First Survey of Eye-Tracking Applications," Behavior Research Methods Instruments Computers J. Psychonomic Soc. Inc., vol. 34, pp. 455-470, 2002.
[25] G.E. Legge, T.S. Klitz, and B. Tjan, "Mr. Chips: An Ideal-Observer Model of Reading," Psychological Rev., vol. 104, pp. 524-553, 1997.
[26] R.D. Rimey and C.M. Brown, "Controlling Eye Movements with Hidden Markov Models," Int'l J. Computer Vision, vol. 7, no. 1, pp. 47-65, 1991.
[27] S. Treue, "Neural Correlates of Attention in Primate Visual Cortex," Trends in Neurosciences, vol. 24, no. 5, pp. 295-300, 2001.
[28] S. Kastner and L.G. Ungerleider, "Mechanisms of Visual Attention in the Human Cortex," Ann. Rev. Neurosciences, vol. 23, pp. 315-341, 2000.
[29] E.T. Rolls and G. Deco, "Attention in Natural Scenes: Neurophysiological and Computational Bases," Neural Networks, vol. 19, no. 9, pp. 1383-1394, 2006.
[30] G.A. Carpenter and S. Grossberg, "A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine," J. Computer Vision, Graphics, and Image Processing, vol. 37, no. 1, pp. 54-115, 1987.
[31] N. Ouerhani and H. Hügli, "Real-Time Visual Attention on a Massively Parallel SIMD Architecture," Real-Time Imaging, vol. 9, no. 3, pp. 189-196, 2003.
[32] Q. Ma, L. Zhang, and B. Wang, "New Strategy for Image and Video Quality Assessment," J. Electronic Imaging, vol. 19, pp. 1-14, 2010.
[33] Y. Ma, X. Hua, L. Lu, and H. Zhang, "A Generic Framework of User Attention Model and Its Application in Video Summarization," IEEE Trans. Multimedia, vol. 7, no. 5, pp. 907-919, Oct. 2005.
[34] A. Ninassi, O. Le Meur, P. Le Callet, and D. Barba, "Does Where You Gaze on an Image Affect Your Perception of Quality? Applying Visual Attention to Image Quality Metric," Proc. IEEE Int'l Conf. Image Processing, vol. 2, pp. 169-172, 2007.
[35] D. Walther and C. Koch, "Modeling Attention to Salient Proto-Objects," Neural Networks, vol. 19, no. 9, pp. 1395-1407, 2006.
[36] C. Siagian and L. Itti, "Biologically Inspired Mobile Robot Vision Localization," IEEE Trans. Robotics, vol. 25, no. 4, pp. 861-873, Aug. 2009.
[37] S. Frintrop and P. Jensfelt, "Attentional Landmarks and Active Gaze Control for Visual SLAM," IEEE Trans. Robotics, vol. 24, no. 5, pp. 1054-1065, Oct. 2008.
[38] D. DeCarlo and A. Santella, "Stylization and Abstraction of Photographs," ACM Trans. Graphics, vol. 21, no. 3, pp. 769-776, 2002.
[39] L. Itti, "Automatic Foveation for Video Compression Using a Neurobiological Model of Visual Attention," IEEE Trans. Image Processing, vol. 13, no. 10, pp. 1304-1318, Oct. 2004.
[40] L. Marchesotti, C. Cifarelli, and G. Csurka, "A Framework for Visual Saliency Detection with Applications to Image Thumbnailing," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[41] O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, "A Coherent Computational Approach to Model Bottom-Up Visual Attention," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 5, pp. 802-817, May 2006.
[42] G. Fritz, C. Seifert, L. Paletta, and H. Bischof, "Attentive Object Detection Using an Information Theoretic Saliency Measure," Proc. Second Int'l Conf. Attention and Performance in Computational Vision, pp. 29-41, 2005.
[43] T. Liu, J. Sun, N.N Zheng, and H.Y Shum, "Learning to Detect a Salient Object," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[44] V. Setlur, R. Raskar, S. Takagi, M. Gleicher, and B. Gooch, "Automatic Image Retargeting, In Mobile and Ubiquitous Multimedia (MUM)," Proc. Fourth Int'l Conf. Mobile and Ubiquitous Multimedia, 2005.
[45] C. Chamaret and O. Le Meur, "Attention-Based Video Reframing: Validation Using Eye-Tracking," Proc. 19th Int'l Conf. Pattern Recognition, 2008.
[46] S. Goferman, L. Zelnik-Manor, and A. Tal, "Context-Aware Saliency Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[47] N. Sadaka and L.J. Karam, "Efficient Perceptual Attentive Super-Resolution," Proc. 16th IEEE Int'l Conf. Image Processing, 2009.
[48] H. Liu, S. Jiang, Q. Huang, and C. Xu, "A Generic Virtual Content Insertion System Based on Visual Attention Analysis," Proc. ACM Int'l Conf. Multimedia, pp. 379-388, 2008.
[49] S. Marat, M. Guironnet, and D. Pellerin, "Video Summarization Using a Visual Attention Model," Proc. 15th European Signal Processing Conf., 2007.
[50] S. Frintrop, VOCUS: A Visual Attention System for Object Detection and Goal-Directed Search. Springer, 2006.
[51] V. Navalpakkam and L. Itti, "An Integrated Model of Top-Down and Bottom-Up Attention for Optimizing Detection Speed," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[52] A. Salah, E. Alpaydin, and L. Akrun, "A Selective Attention-Based Method for Visual Pattern Recognition with Application to Handwritten Digit Recognition and Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 420-425, Mar. 2002.
[53] S. Frintrop, "General Object Tracking with a Component-Based Target Descriptor," Proc. IEEE Int'l Conf. Robotics and Automation, pp. 4531-4536, 2010.
[54] M.S. El-Nasr, T. Vasilakos, C. Rao, and J. Zupko, "Dynamic Intelligent Lighting for Directing Visual Attention in Interactive 3D Scenes," IEEE Trans. Computational Intelligence and AI in Games, vol. 1, no. 2, pp. 145-153, June 2009.
[55] G. Boccignone, "Nonparametric Bayesian Attentive Video Analysis," Proc. 19th Int'l Conf. Pattern Recognition, 2008.
[56] G. Boccignone, A. Chianese, V. Moscato, and A. Picariello, "Foveated Shot Detection for Video Segmentation," IEEE Trans. Circuits and Systems for Video Technology, vol. 15, no. 3, pp. 365-377, Mar. 2005.
[57] B. Mertsching, M. Bollmann, R. Hoischen, and S. Schmalz, "The Neural Active Vision System," Handbook of Computer Vision and Applications, Academic Press, 1999.
[58] A. Dankers, N. Barnes, and A. Zelinsky, "A Reactive Vision System: Active-Dynamic Saliency," Proc. Int'l Conf. Vision Systems, 2007.
[59] N. Ouerhani, A. Bur, and H. Hügli, "Visual Attention-Based Robot Self-Localization," Proc. European Conf. Mobile Robotics, pp. 803-813, 2005.
[60] S. Baluja and D. Pomerleau, "Expectation-Based Selective Attention for Visual Monitoring and Control of a Robot Vehicle," Robotics and Autonomous Systems, vol. 22, nos. 3/4, pp. 329-344, 1997.
[61] C. Scheier and S. Egner, "Visual Attention in a Mobile Robot," Proc. Int'l Symp. Industrial Electronics, pp. 48-53, 1997.
[62] C. Breazeal, "A Context-Dependent Attention System for a Social Robot," Proc. 16th Int'l Joint Conf. Artificial Intelligence, pp. 1146-1151, 1999.
[63] G. Heidemann, R. Rae, H. Bekel, I. Bax, and H. Ritter, "Integrating Context-Free and Context-Dependent Attentional Mechanisms for Gestural Object Reference," Machine Vision Application, vol. 16, no. 1, pp. 64-73, 2004.
[64] G. Heidemann, "Focus-of-Attention from Local Color Symmetries," IEEE Trans Pattern Analysis and Machine Intelligence, vol. 26, no. 7, pp. 817-830, July 2004.
[65] A. Belardinelli, "Salience Features Selection: Deriving a Model from Human Evidence," PhD thesis, 2008.
[66] Y. Nagai, "From Bottom-up Visual Attention to Robot Action Learning," Proc. Eighth IEEE Int'l Conf. Development and Learning, 2009.
[67] C. Muhl, Y. Nagai, and G. Sagerer, "On Constructing a Communicative Space in HRI," Proc. 30th German Conf. Artificial Intelligence, 2007.
[68] T. Liu, S.D. Slotnick, J.T. Serences, and S. Yantis, "Cortical Mechanisms of Feature-Based Intentional Control," Cerebral Cortex, vol. 13, no. 12, pp. 1334-1343, 2003.
[69] B.W. Hong and M. Brady, "A Topographic Representation for Mammogram Segmentation," Proc. Medical Image Computing and Computer Assisted Intervention, pp. 730-737, 2003.
[70] N. Parikh, L. Itti, and J. Weiland, "Saliency-Based Image Processing for Retinal Prostheses," J. Neural Eng., vol 7, no 1, pp. 1-10, 2010.
[71] O.R. Joubert, D. Fize, G.A. Rousselet, and M. Fabre-Thorpe, "Early Interference of Context Congruence on Object Processing in Rapid Visual Categorization of Natural Scenes," J. Vision, vol. 8, no. 13, pp. 1-18, 2008.
[72] H. Li and K.N. Ngan, "Saliency Model-Based Face Segmentation and Tracking in Head-and-Shoulder Video Sequences," J. Vision Comm. and Image Representation, vol. 19, pp. 320-333, 2008.
[73] N. Courty and E. Marchand, "Visual Perception Based on Salient Features," Proc. Int'l Conf. Intelligent Robots and Systems, 2003.
[74] F. Shic and B. Scassellati, "A Behavioral Analysis of Computational Models of Visual Attention," Int'l J. Computer Vision, vol. 73, pp. 159-177, 2007.
[75] H.C. Nothdurft, "Salience of Feature Contrast," Neurobiology of Attention, L. Itti, G. Rees, and J. K. Tsotsos, eds., Academic Press, 2005.
[76] M. Corbetta and G.L. Shulman, "Control of Goal-Directed and Stimulus-Driven Attention in the Brain," Natural Rev., vol. 3, no. 3, pp. 201-215, 2002.
[77] L. Itti and C. Koch, "Computational Modeling of Visual Attention," Natural Rev. Neuroscience, vol. 2, no. 3, pp. 194-203, 2001.
[78] H.E. Egeth and S. Yantis, "Visual Attention: Control, Representation, and Time Course," Ann. Rev. Psychologogy, vol. 48, pp. 269-297, 1997.
[79] A.L. Yarbus, Eye-Movements and Vision. Plenum Press, 1967.
[80] V. Navalpakkam and L. Itti, "Modeling the Influence of Task on Attention," Vision Research, vol. 45, no. 2, pp. 205-231, 2005.
[81] A.M. Treisman and G. Gelade, "A Feature Integration Theory of Attention," Cognitive Psychology, vol. 12, pp. 97-136, 1980.
[82] J.M. Wolfe, "Guided Search 4.0: Current Progress with a Model of Visual Search," Integrated Models of Cognitive Systems, W.D. Gray, ed., Oxford Univ. Press, 2007.
[83] G.J. Zelinsky, "A Theory of Eye Movements during Target Acquisition," Psychological Rev., vol. 115, no. 4, pp. 787-835, 2008.
[84] W. Einhauser, M. Spain, and P. Perona, "Objects Predict Fixations Better Than Early Saliency," J. Vision, vol. 14, pp. 1-26, 2008.
[85] M. Pomplun, "Saccadic Selectivity in Complex Visual Search Displays," Vision Research, vol. 46, pp. 1886-1900, 2006.
[86] A. Hwang and M. Pomplun, "A Model of Top-Down Control of Attention during Visual Search in Real-World Scenes," J. Vision, vol. 8, no. 6, Article 681, 2008.
[87] K. Ehinger, B. Hidalgo-Sotelo, A. Torralba, and A. Oliva, "Modeling Search for People in 900 Scenes: A Combined Source Model of Eye Guidance," Visual Cognition, vol. 17, pp. 945-978, 2009.
[88] A. Borji, M.N. Ahmadabadi, B.N. Araabi, and M. Hamidi, "Online Learning of Task-Driven Object-Based Visual Attention Control," J. Image and Vision Computing, vol. 28, pp. 1130-1145, 2010.
[89] A. Borji, M.N. Ahmadabadi, and B.N. Araabi, "Cost-Sensitive Learning of Top-Down Modulation for Attentional Control," Machine Vision and Applications, vol. 22, pp. 61-76, 2011.
[90] L. Elazary and L. Itti, "A Bayesian Model for Efficient Visual Search and Recognition," Vision Research, vol. 50, pp. 1338-1352, 2010.
[91] M.M. Chun and Y. Jiang, "Contextual Cueing: Implicit Learning and Memory of Visual Context Guides Spatial Attention," Cognitive Psychology, vol. 36, pp. 28-71, 1998.
[92] A. Torralba, "Modeling Global Scene Factors in Attention," J. Optical Soc. Am., vol. 20, no. 7, pp. 1407-1418, 2003.
[93] A. Oliva and A. Torralba, "Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope," Int'l J. Computer Vision, vol. 42, pp. 145-175, 2001.
[94] L.W. Renninger and J. Malik, "When Is Scene Recognition Just Texture Recognition?" Vision Research, vol. 44, pp. 2301-2311, 2004.
[95] C. Siagian and L. Itti, "Rapid Biologically-Inspired Scene Classification Using Features Shared with Visual Attention," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 300-312, Feb. 2007.
[96] M. Viswanathan, C. Siagian, and L. Itti, Vision Science Symp., 2007.
[97] J. Triesch, D.H. Ballard, M.M. Hayhoe, and B.T. Sullivan, "What You See Is What You Need," J. Vision, vol. 3, pp. 86-94, 2003.
[98] M.I. Posner, "Orienting of Attention," Quarterly J. Experimental Psychology, vol. 32, pp. 3-25, 1980.
[99] M. Hayhoe and D. Ballard, "Eye Movements in Natural Behavior," Trends in Cognitive Sciences, vol. 9, pp. 188-194, 2005.
[100] M.S. Mirian, M.N. Ahmadabadi, B.N. Araabi, R.R. Siegwart, "Learning Active Fusion of Multiple Experts' Decisions: An Attention-Based Approach," Neural Computation, 2011.
[101] R.J. Peters and L. Itti, "Beyond Bottom-up: Incorporating Task-dependent Influences into a Computational Model of Spatial Attention," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[102] D. Pang, A. Kimura, T. Takeuchi, J. Yamato, and K. Kashino, "A Stochastic Model of Selective Visual Attention with a Dynamic Bayesian Network," Proc. IEEE Int'l Conf. Multimedia and Expo., 2008.
[103] Y. Zhai and M. Shah, "Visual Attention Detection in Video Sequences Using Spatiotemporal Cues," Proc. ACM Int'l Conf. Multimedia, 2006.
[104] S. Marat, T. Ho-Phuoc, L. Granjon, N. Guyader, D. Pellerin, and A. Guérin-Dugué, "Modeling Spatio-Temporal Saliency to Predict Gaze Direction for Short Videos," Int'l J. Computer Vision, vol. 82, pp. 231-243, 2009.
[105] V. Mahadevan and N. Vasconcelos, "Spatiotemporal Saliency in Dynamic Scenes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 171-177, Jan. 2010.
[106] V. Mahadevan and N. Vasconcelos, "Saliency Based Discriminant Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[107] N. Jacobson, Y-L. Lee, V. Mahadevan, N. Vasconcelos, and T.Q. Nguyen, "A Novel Approach to FRUC Using Discriminant Saliency and Frame Segmentation," IEEE Trans. Image Processing, vol. 19, no. 11, pp. 2924-2934, Nov. 2010.
[108] H.J. Seo and P. Milanfar, "Static and Space-Time Visual Saliency Detection by Self-Resemblance," J. Vision, vol. 9, no. 12, pp. 1-27, 2009.
[109] N. Sprague and D.H. Ballard, "Eye Movements for Reward Maximization," Proc. Advances in Neural Information Processing, 2003.
[110] http://tcts.fpms.ac.be / mousetrack/ 2012.
[111] J. Bisley and M. Goldberg, "Neuronal Activity in the Lateral Intraparietal Area and Spatial Attention," Science, vol. 299, pp. 81-86, 2003.
[112] J. Duncan, "Selective Attention and the Organization of Visual Information," J. Experimental Psychology, vol. 113, pp. 501-517, 1984.
[113] B.J. Scholl, "Objects and Attention: The State of the Art," Cognition, vol. 80, pp. 1-46, 2001.
[114] Z.W. Pylyshyn and R.W. Storm, "Tracking Multiple Independent Targets: Evidence for a Parallel Tracking Mechanism," Spatial Vision, vol. 3, pp. 179-197, 1988.
[115] E. Awh and H. Pashler, "Evidence for Split Attentional Foci," J. Experimental Psychology Human Perception and Performance, vol. 26, pp. 834-846, 2000.
[116] B.C. Russell, A. Torralba, K.P. Murphy, and W.T. Freeman, "LabelMe: A Database and Web-Based Tool for Image Annotation," Int'l J. Computer Vision, vol. 77, nos. 1-3, pp. 157-173, 2008.
[117] Y. Sun and R. Fisher, "Object-Based Visual Attention for Computer Vision," Artificial Intelligence, vol. 146, no. 1, pp. 77-123, 2003.
[118] J.M. Wolfe and T.S. Horowitz, "What Attributes Guide the Deployment of Visual Attention and How Do They Do It?" Natural Rev. Neuroscience, vol. 5, pp. 1-7, 2004.
[119] L. Itti, N. Dhavale, and F. Pighin, "Realistic Avatar Eye and Head Animation Using a Neurobiological Model of Visual Attention," Proc. SPIE, vol. 5200, pp. 64-78, 2003.
[120] R. Rae, "Gestikbasierte Mensch-Maschine-Kommunikation auf der Grundlage Visueller Aufmerksamkeit und Adaptivität," PhD thesis, Universität Bielefeld, 2000.
[121] J. Harel, C. Koch, and P. Perona, "Graph-Based Visual Saliency," Neural Information Processing Systems, vol. 19, pp. 545-552, 2006.
[122] O. Boiman and M. Irani, "Detecting Irregularities in Images and in Video," Proc. IEEE Int'l Conf. Computer Vision, 2005.
[123] B.W. Tatler, "The Central Fixation Bias in Scene Viewing: Selecting an Optimal Viewing Position Independently of Motor Bases and Image Feature Distributions," J. Vision, vol. 14, pp. 1-17, 2007.
[124] R. Milanese, "Detecting Salient Regions in an Image: From Biological Evidence to Computer Implementation," PhD thesis, Univ. Geneva, 1993.
[125] F.H. Hamker, "The Emergence of Attention by Poulation-based Inference and Its Role in Distributed Processing and Cognitive Control of Vision," J. Computer Vision Image Understanding, vol. 100, nos. 1/2, pp. 64-106, 2005.
[126] S. Vijayakumar, J. Conradt, T. Shibata, and S. Schaal, "Overt Visual Attention For a Humanoid Robot," Proc. IEEE/RSJ Int'l Conf. Intelligent Robots and Systems, 2001.
[127] C.M. Privitera and L.W. Stark, "Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 9, pp. 970-982, Sept. 2000.
[128] K. Lee, H. Buxton, and J. Feng, "Selective Attention for Cue-guided Search Using a Spiking Neural Network," Proc. Int'l Workshop Attention and Performance in Computer Vision, p. 5562, 2003.
[129] T. Kadir and M. Brady, "Saliency, Scale and Image Description," Int'l J. Computer Vision, vol. 45, no. 2, pp. 83-105, 2001.
[130] A. Maki, P. Nordlund, and J.O. Eklundh, "Attentional Scene Segmentation: Integrating Depth and Motion," Computer Vision and Image Understanding, vol. 78, no. 3, pp. 351-373, 2000.
[131] D. Parkhurst, K. Law, and E. Niebur, "Modeling the Role of Salience in the Allocation of Overt Visual Attention," Vision Research, vol. 42, nos. 1, pp. 107-123, 2002.
[132] T.S. Horowitz and J.M. Wolfe, "Visual Search Has No Memory," Nature, vol. 394, pp. 575-577, 1998.
[133] J. Li, Y. Tian, T. Huang, and W. Gao, "Probabilistic Multi-Task Learning for Visual Saliency Estimation in Video," Int'l J. Computer Vision, vol. 90, pp. 150-165, 2010.
[134] R. Peters, A. Iyer, L. Itti, and C. Koch, "Components of Bottom-Up Gaze Allocation in Natural Images," Vision Research, vol. 45, pp. 2397-2416, 2005.
[135] M. Land and M. Hayhoe, "In What Ways Do Eye Movements Contribute to Everyday Activities?" Vision Research, vol. 41, pp. 3559-3565, 2001.
[136] G. Kootstra, A. Nederveen, and B. de Boer, "Paying Attention to Symmetry," Proc. British Machine Vision Conf., pp. 1115-1125, 2008.
[137] D. Reisfeld, H. Wolfson, and Y. Yeshurun, "Context-Free Attentional Operators: The Generalized Symmetry Transform," Int'l J. Computer Vision, vol. 14, no. 2, pp. 119-130, 1995.
[138] O. Le Meur, P. Le Callet, and D. Barba, "Predicting Visual Fixations on Video Based on Low-Level Visual Features," Vision Research, vol. 47/19, pp. 2483-2498, 2007.
[139] D.D. Salvucci, "An Integrated Model of Eye Movements and Visual Encoding," Cognitive Systems Research, vol. 1, pp. 201-220, 2001.
[140] A. Oliva, A. Torralba, M.S. Castelhano, and J.M. Henderson, "Top-Down Control of Visual Attention in Object Detection," Proc. Int'l Conf. Image Processing, pp. 253-256, 2003.
[141] L. Zhang, M.H. Tong, T.K. Marks, H. Shan, and G.W. Cottrell, "SUN: A Bayesian Framework for Saliency Using Natural Statistics," J. Vision, vol. 8, no. 32, pp. 1-20, 2008.
[142] L. Zhang, M.H. Tong, and G.W. Cottrell, "SUNDAy: Saliency Using Natural Statistics for Dynamic Analysis of Scenes," Proc. 31st Ann. Cognitive Science Soc. Conf., 2009.
[143] N.D.B. Bruce and J.K. Tsotsos, "Spatiotemporal Saliency: Towards a Hierarchical Representation of Visual Saliency," Proc. Int'l Workshop Attention in Cognitive Systems, 2008.
[144] N.D.B. Bruce and J.K. Tsotsos, "Saliency Based on Information Maximization," Proc. Advances in Neural Information Processing Systems, 2005.
[145] L. Itti and P. Baldi, "Bayesian Surprise Attracts Human Attention," Proc. Advances in Neural Information Processing Systems, 2005.
[146] D. Gao and N. Vasconcelos, "Discriminant Saliency for Visual Recognition from Cluttered Scenes," Proc. Advances in Neural Information Processing Systems, 2004.
[147] D. Gao, S. Han, and N. Vasconcelos, "Discriminant Saliency, the Detection of Suspicious Coincidences, and Applications to Visual Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 6, pp. 989-1005, June 2009.
[148] E. Gu, J. Wang, and N.I. Badler, "Generating Sequence of Eye Fixations Using Decision-Theoretic Attention Model," Proc. Workshop Attention and Performance in Computational Vision, pp. 277-29, 2007.
[149] T.S. Lee and S. Yu, "An Information-Theoretic Framework for Understanding Saccadic Behaviors," Proc. Advanced in Neural Processing Systems, 2000.
[150] X. Hou and L. Zhang, "Saliency Detection: A Spectral Residual Approach," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[151] X. Hou and L. Zhang, "Dynamic Visual Attention: Searching for Coding Length Increments," Proc. Advances in Neural Information Processing Systems, pp. 681-688, 2008.
[152] M. Mancas, "Computational Attention: Modelisation and Application to Audio and Image Processing," PhD thesis, 2007.
[153] T. Avraham and M. Lindenbaum, "Esaliency (Extended Saliency): Meaningful Attention Using Stochastic Image Modeling," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 4, pp. 693-708, Apr. 2010.
[154] S. Chikkerur, T. Serre, C. Tan, and T. Poggio, "What and Where: A Bayesian Inference Theory of Visual Attention," Vision Research, vol. 55, pp. 2233-2247, 2010.
[155] P. Verghese, "Visual Search and Attention: A Signal Detection Theory Approach," Neuron, vol. 31, pp. 523-535, 2001.
[156] C. Guo, Q. Ma, and L. Zhang, "Spatio-Temporal Saliency Detection Using Phase Spectrum of Quaternion Fourier Transform," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[157] C. Guo and L. Zhang, "A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression," IEEE Trans. Image Processing, vol. 19, no. 1, pp. 185-198, Jan. 2010.
[158] R. Achanta, S.S. Hemami, F.J. Estrada, and S. Süsstrunk, "Frequency-Tuned Salient Region Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[159] P. Bian and L. Zhang, "Biological Plausibility of Spectral Domain Approach for Spatiotemporal Visual Saliency," Proc. 15th Int'l Conf. Advances in Neuro-Information Processing, pp. 251-258, 2009.
[160] A. Garcia-Diaz, X.R. Fdez-Vidal, X.M. Pardo, and R. Dosil, "Decorrelation and Distinctiveness Provide with Human-Like Saliency," Proc. Advanced Concepts for Intelligent Vision Systems, pp. 343-354, 2009.
[161] N.J. Butko and J.R. Movellan, "Optimal Scanning for Faster Object Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[162] S. Jodogne and J. Piater, "Closed-Loop Learning of Visual Control Policies," J. Artificial Intelligence Research, vol. 28, pp. 349-391, 2007.
[163] R. McCallum, "Reinforcement Learning with Selective Perception and Hidden State," PhD thesis, 1996.
[164] L. Paletta, G. Fritz, and C. Seifert, "Q-Learning of Sequential Attention for Visual Object Recognition from Informative Local Descriptors," Proc. 22nd Int'l Conf. Machine Learning, pp. 649-656, 2005.
[165] W. Kienzle, M.O. Franz, B. Schölkopf, and F.A. Wichmann, "Center-Surround Patterns Emerge as Optimal Predictors for Human Saccade Targets," J. Vision, vol. 9, pp. 1-15, 2009.
[166] T. Judd, K. Ehinger, F. Durand, and A. Torralba, "Learning to Predict Where Humans Look," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[167] M. Cerf, J. Harel, W. Einhäuser, and C. Koch, "Predicting Human Gaze Using Low-Level Saliency Combined with Face Detection," Advances in Neural Information Processing Systems, vol. 20, pp. 241-248, 2007.
[168] O. Ramström and H.I. Christensen, "Visual Attention Using Game Theory," Proc. Biologically Motivated Computer Vision Conf., pp. 462-471, 2002.
[169] P.L. Rosin, "A Simple Method for Detecting Salient Regions," Pattern Recognition, vol. 42, no. 11, pp. 2363-2371, 2009.
[170] Z. Li, "A Saliency Map in Primary Visual Cortex," Trends in Cognitive Sciences, vol. 6, no. 1, pp. 9-16, 2002.
[171] Y. Li, Y. Zhou, J. Yan, and J. Yang, "Visual Saliency Based on Conditional Entropy," Proc. Ninth Asian Conf. Computer Vision, 2009.
[172] S.W Ban, I. Lee, and M. Lee, "Dynamic Visual Selective Attention Model," Neurocomputing, vol. 71, nos. 4-6, pp. 853-856, 2008.
[173] M.T. López, M.A. Fernńdez, A. Fernández-Caballero, J. Mira, and A.E. Delgado, "Dynamic Visual Attention Model in Image Sequences," J. Image and Vision Computing, vol. 25, pp. 597-613, 2007.
[174] U. Rajashekar, I. van der Linde, A.C. Bovik, and L.K. Cormack, "GAFFE: A Gaze-Attentive Fixation Finding Engine," IEEE Trans. Image Processing, vol. 17, no. 4, pp. 564-573, Apr. 2008.
[175] G. Boccignone and M. Ferraro, "Modeling Gaze Shift as a Constrained Random Walk," Physica A, vol. 331, 2004.
[176] M.C. potter, "Meaning in Visual Scenes," Science, vol. 187, pp. 965-966, 1975.
[177] J.M. Henderson and A. Hollingworth, "High-Level Scene Perception," Ann. Rev. Psychology, vol. 50, pp. 243-271, 1999.
[178] R.A. Rensink, "The Dynamic Representation of Scenes," Visual Cognition, vol. 7, pp. 17-42, 2000.
[179] J. Bailenson and N. Yee, "Digital Chameleons: Automatic Assimilation of Nonverbal Gestures in Immersive Virtual Environments," Psychological Science, vol. 16, pp. 814-819, 2005.
[180] M. Sodhi, B. Reimer, J.L. Cohen, E. Vastenburg, R. Kaars, and S. Kirschenbaum, "On-Road Driver Eye Movement Tracking Using Head-Mounted Devices," Proc. Symp. Eye Tracking Research and Applications, 2002.
[181] J.H. Reynolds and D.J. Heeger, "The Normalization Model of Attention," Neuron, vol. 61, no. 2, pp. 168-185, 2009.
[182] S. Engmann, B.M. Hart, T. Sieren, S. Onat, P. König, and W. Einhäuser, "Saliency on a Natural Scene Background: Effects of Color and Luminance Contrast Add Linearly," Attention, Perception and Psychophysics, vol. 71, no. 6, pp. 1337-1352, 2009.
[183] A. Reeves and G. Sperling, "Attention Gating in Short-Term Visual Memory," Psychological Rev., vol. 93, no. 2, pp. 180-206, 1986.
[184] L. Itti, "Quantifying the Contribution of Low-Level Saliency to Human Eye Movements in Dynamic Scenes," Visual Cognition, vol. 12, no. 6, pp. 1093-1123, 2005.
[185] D. Gao, V. Mahadevan, and N. Vasconcelos, "On the Plausibility of the Discriminant Center-Surround Hypothesis for Visual Saliency," J. Vision, vol. 8, nos. 7-13, pp. 1-18, 2008.
[186] J. Yan, J. Liu, Y. Li, and Y. Liu, "Visual Saliency via Sparsity Rank Decomposition," Proc. IEEE 17th Int'l Conf. Image Processing, 2010.
[187] http://www.its.caltech.edu~xhou/ 2012.
[188] J. Yuen, B.C. Russell, C. Liu, and A. Torralba, "LabelMe Video: Building a Video Database with Human Annotations," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[189] R. Rosenholtz, Y. Li, and L. Nakano, "Measuring Visual Clutter," J. Vision, vol. 7, no. 17, pp. 1-22, 2007.
[190] R. Rosenholtz, A. Dorai, and R. Freeman, "Do Predictions of Visual Perception Aid Design?" ACM Trans. Applied Perception, vol. 8, no. 2, Article 12, 2011.
[191] R. Rosenholtz, "A Simple Saliency Model Predicts a Number of Motion Popout Phenomena," Vision Research, vol. 39, pp. 3157-3163, 1999.
[192] X. Hou, J. Harel, and C. Koch, "Image Signature: Highlighting Sparse Salient Regions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 1, pp. 194-201, Jan. 2012.
[193] R. Rosenholtz, A.L. Nagy, and N.R. Bell, "The Effect of Background Color on Asymmetries in Color Search," J. Vision, vol. 4, no. 3, pp. 224-240, 2004.
[194] http://alpern.mit.edusaliency/ 2012.
[195] D. Green and J. Swets, Signal Detection Theory and Psychophysics. John Wiley, 1966.
[196] T. Jost, N. Ouerhani, R. von Wartburg, R. Mäuri, and H. Häugli, "Assessing the Contribution of Color in Visual Attention," Computer Vision and Image Understanding, vol. 100, pp. 107-123, 2005.
[197] U. Rajashekar, A.C. Bovik, and L.K. Cormack, "Visual Search in Noise: Revealing the Influence of Structural Cues by Gaze-Contingent Classification Image Analysis," J. Vision, vol. 13, pp. 379-386, 2006.
[198] S.A. Brandt and L.W. Stark, "Spontaneous Eye Movements during Visual Imagery Reflect the Content of the Visual Scene," J. Cognitive Neuroscience, vol. 9, nos. 27-38, pp. 27-38, 1997.
[199] A.D. Hwang, H.C. Wang, and M. Pomplun, "Semantic Guidance of Eye Movements in Real-World Scenes," Vision Research, vol. 51, pp. 1192-1205, 2011.
[200] N. Murray, M. Vanrell, X. Otazu, and C. Alejandro Parraga, "Saliency Estimation Using a Non-Parametric Low-Level Vision Model," Proc. IEEE Computer Vision and Pattern Recognition, 2011.
[201] W. Wang, C. Chen, Y. Wang, T. Jiang, F. Fang, and Y. Yao, "Simulating Human Saccadic Scanpaths on Natural Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[202] R.L Canosa, "Real-World Vision: Selective Perception and Task," ACM Trans. Applied Perception, vol. 6, no. 2, Article 11, 2009.
[203] M.S. Peterson, A.F. Kramer, and D.E. Irwin, "Covert Shifts of Attention Precede Involuntary Eye Movements," Perception and Psychophysics, vol. 66, pp. 398-405, 2004.
[204] F. Baluch and L. Itti, "Mechanisms of Top-Down Attention," Trends in Neuroscience, vol. 34, no. 4, pp. 210-24, 2011.
[205] J. Hayes and A. Efros, "Scene Completion Using Millions of Photographs," Proc. ACM Siggraph, 2007.
[206] P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan, "Object Detection with Discriminatively Trained Part Based Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1627-1645, Apr. 2010.
[207] A.K. Mishra and Y. Aloimonos, "Active Segmentation," Int'l J. Humanoid Robotics, vol 6, pp 361-386, 2009.
[208] B. Suh, H. Lingm, B.B. Bederson, and D.W. Jacobs, "Automatic Thumbnail Cropping and Its Effectiveness," Proc. 16th Ann. ACM Symp. User Interface Software and Technology, pp. 95-104, 2003.
[209] S. Mitri, S. Frintrop, K. Pervolz, H. Surmann, and A. Nuchter, "Robust Object Detection at Regions of Interest with an Application in Ball Recognition," Proc. IEEE Int'l Conf. Robotics and Animation, pp. 126-131, Apr. 2005.
[210] N. Ouerhani, R. von Wartburg, H. Hugli, and R.M. Muri, "Empirical Validation of Saliency-Based Model of Visual Attention," Electronic Letters Computer Vision and Image Analysis, vol. 3, no. 1, pp. 13-24, 2003.
[211] L.W. Stark and Y. Choi, "Experimental Metaphysics: The Scanpath as an Epistemological Mechanism," Visual Attention and Cognition, pp. 3-69, 1996.
[212] P. Reinagel and A. Zador, "Natural Scenes at the Center of Gaze," Network, vol. 10, pp. 341-50, 1999.
[213] U. Engelke, H.J. Zepernick, and A. Maeder, "Visual Attention Modeling: Region-of-Interest Versus Fixation Patterns," Proc. Picture Coding Symp., 2009.
[214] M. Verma and P.W. McOwana, "Generating Customised Experimental Stimuli for Visual Search Using Genetic Algorithms Shows Evidence for a Continuum of Search Efficiency," Vision Research, vol. 49, no. 3, pp. 374-382, 2009.
[215] S. Han and N. Vasconcelos, "Biologically Plausible Saliency Mechanisms Improve Feedforward Object Recognition," Vision Research, vol. 50, no. 22, pp. 2295-2307, 2010.
[216] D. Ballard, M. Hayhoe, and J. Pelz, "Memory Representations in Natural Tasks," J. Cognitive Neuroscience, vol. 7, no. 1, pp. 66-80, 1995.
[217] R. Rao, "Bayesian Inference and Attentional Modulation in the Visual Cortex," NeuroReport, vol. 16, no. 16, pp. 1843-1848, 2005.
[218] A.Borji D.N. Sihite and L. Itti, "Computational Modeling of Top-Down Visual Attention in Interactive Environments," Proc. British Machine Vision Conf., 2011.
[219] E. Niebur and C. Koch, "Control of Selective Visual Attention: Modeling the Where Pathway," Proc. Advances in Neural Information Processing Systems, pp. 802-808, 1995.
[220] P. Viola and M.J. Jones, "Robust Real-Time Face Detection," Int'l J. Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[221] W. Kienzle, B. Schølkopf, F.A. Wichmann, and M.O. Franz, "How to Find Interesting Locations in Video: A Spatiotemporal Interest Point Detector Learned from Human Eye Movements," Proc. 29th DAGM Conf. Pattern Recognition, pp. 405-414, 2007.
[222] J. Wang, J. Sun, L. Quan, X. Tang, and H.Y Shum, "Picture Collage," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[223] D. Gao and N. Vasconcelos, "Decision-Theoretic Saliency: Computational Principles, Biological Plausibility, and Implications for Neurophysiology and Psychophysics," Neural Computation, vol. 21, pp. 239-271, 2009.
[224] M. Carrasco, "Visual Attention: The Past 25 Years," Vision Research, vol. 51, pp. 1484-1525, 2011.
7 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool