Issue No. 05 - May (2011 vol. 33)
Yiming Liu , Nanyang Technological University, Singapore
Ivor Wai-Hung Tsang , Nanyang Technological University, Singapore
Jiebo Luo , Kodak Research Laboratories, Eastman Kodak Company, Rochester
Dong Xu , Nanyang Technological University, Singapore
The rapid popularization of digital cameras and mobile phone cameras has led to an explosive growth of personal photo collections by consumers. In this paper, we present a real-time textual query-based personal photo retrieval system by leveraging millions of Web images and their associated rich textual descriptions (captions, categories, etc.). After a user provides a textual query (e.g., "water”), our system exploits the inverted file to automatically find the positive Web images that are related to the textual query "water” as well as the negative Web images that are irrelevant to the textual query. Based on these automatically retrieved relevant and irrelevant Web images, we employ three simple but effective classification methods, k-Nearest Neighbor (kNN), decision stumps, and linear SVM, to rank personal photos. To further improve the photo retrieval performance, we propose two relevance feedback methods via cross-domain learning, which effectively utilize both the Web images and personal images. In particular, our proposed cross-domain learning methods can learn robust classifiers with only a very limited amount of labeled personal photos from the user by leveraging the prelearned linear SVM classifiers in real time. We further propose an incremental cross-domain learning method in order to significantly accelerate the relevance feedback process on large consumer photo databases. Extensive experiments on two consumer photo data sets demonstrate the effectiveness and efficiency of our system, which is also inherently not limited by any predefined lexicon.
Textual query-based consumer photo retrieval, large-scale Web data, cross-domain learning.
Yiming Liu, Ivor Wai-Hung Tsang, Jiebo Luo, Dong Xu, "Textual Query of Personal Photos Facilitated by Large-Scale Web Data", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 33, no. , pp. 1022-1036, May 2011, doi:10.1109/TPAMI.2010.142