The Community for Technology Leaders
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
Boston, MA, USA
June 7, 2015 to June 12, 2015
ISSN: 1063-6919
ISBN: 978-1-4673-6963-3
pp: 1072-1080
Ming Jiang , Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117583
Shengsheng Huang , Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117583
Juanyong Duan , Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117583
Qi Zhao , Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117583
ABSTRACT
Saliency in Context (SALICON) is an ongoing effort that aims at understanding and predicting visual attention. This paper presents a new method to collect large-scale human data during natural explorations on images. While current datasets present a rich set of images and task-specific annotations such as category labels and object segments, this work focuses on recording and logging how humans shift their attention during visual exploration. The goal is to offer new possibilities to (1) complement task-specific annotations to advance the ultimate goal in visual understanding, and (2) understand visual attention and learn saliency models, all with human attentional data at a much larger scale. We designed a mouse-contingent multi-resolutional paradigm based on neurophysiological and psychophysical studies of peripheral vision, to simulate the natural viewing behavior of humans. The new paradigm allowed using a general-purpose mouse instead of an eye tracker to record viewing behaviors, thus enabling large-scale data collection. The paradigm was validated with controlled laboratory as well as large-scale online data. We report in this paper a proof-of-concept SALICON dataset of human “free-viewing” data on 10,000 images from the Microsoft COCO (MS COCO) dataset with rich contextual information. We evaluated the use of the collected data in the context of saliency prediction, and demonstrated them a good source as ground truth for the evaluation of saliency algorithms.
INDEX TERMS
CITATION

M. Jiang, S. Huang, J. Duan and Q. Zhao, "SALICON: Saliency in Context," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 1072-1080.
doi:10.1109/CVPR.2015.7298710
176 ms
(Ver 3.3 (11022016))