The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - October (2011 vol.33)
pp: 1991-2001
Zhong Wu , Tsinghua University, Beijing
Qifa Ke , Microsoft Research Silicon Valley, Mountain View
Jian Sun , Microsoft Research Asia, Beijing
Heung-Yeung Shum , Microsoft Corporation, Redmond
ABSTRACT
State-of-the-art image retrieval systems achieve scalability by using a bag-of-words representation and textual retrieval methods, but their performance degrades quickly in the face image domain, mainly because they produce visual words with low discriminative power for face images and ignore the special properties of faces. The leading features for face recognition can achieve good retrieval performance, but these features are not suitable for inverted indexing as they are high-dimensional and global and thus not scalable in either computational or storage cost. In this paper, we aim to build a scalable face image retrieval system. For this purpose, we develop a new scalable face representation using both local and global features. In the indexing stage, we exploit special properties of faces to design new component-based local features, which are subsequently quantized into visual words using a novel identity-based quantization scheme. We also use a very small Hamming signature (40 bytes) to encode the discriminative global feature for each face. In the retrieval stage, candidate images are first retrieved from the inverted index of visual words. We then use a new multireference distance to rerank the candidate images using the Hamming signature. On a one millon face database, we show that our local features and global Hamming signatures are complementary—the inverted index based on local features provides candidate images with good recall, while the multireference reranking with global Hamming signature leads to good precision. As a result, our system is not only scalable but also outperforms the linear scan retrieval system using the state-of-the-art face recognition feature in term of the quality.
INDEX TERMS
Face recognition, content-based image retrieval, inverted indexing, image search.
CITATION
Zhong Wu, Qifa Ke, Jian Sun, Heung-Yeung Shum, "Scalable Face Image Retrieval with Identity-Based Quantization and Multireference Reranking", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 10, pp. 1991-2001, October 2011, doi:10.1109/TPAMI.2011.111
REFERENCES
[1] Z. Cao, Q. Yin, J. Sun, and X. Tang, "Face Recognition with Learning-Based Descriptorand Pose-Adaptive Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[2] J.G. Carbonell, Y. Yang, R.E. Frederking, R.D. Brown, Y. Geng, and D. Lee, "Translingual Information Retrieval: A Comparative Evaluation," Proc. 15th Int'l Joint Conf. Artificial Intelligence, pp. 708-714, 1997.
[3] J. Chen, R. Ma, and Z. Su, "Weighting Visual Features with Pseudo Relevance Feedback for cbir," Proc. ACM Int'l Conf. Image and Video Retrieval, 2010.
[4] J. Friedman, J. Bentley, and R. Finkel, "An Algorithm for Finding Best Matches in Logarithmic Expected Time," ACM Trans. Math. Software, vol. 3, pp. 209-226, 1977.
[5] A. Gionis, P. Indyk, and R. Motwani, "Similarity Search in High Dimensions via Hashing," Proc. 25th Int'l Conf. Very Large Data Bases, 1999.
[6] G. Hua and A. Akbarzadeh, "A Robust Elastic and Partial Matching Metric for Face Recognition," Proc. IEEE 12th Int'l Conf. Computer Vision, 2009.
[7] G.B. Huang, M. Mattar, T. Berg, and E. Learned-Miller, "Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments," Proc. European Conf. Computer Vision, 2008.
[8] H. Jegou, M. Douze, and C. Schmid, "Hamming Embedding, and Weak Geometric Consistency for Large Scale Image Search," Proc. 10th European Conf. Computer Vision, 2008.
[9] H. Jegou, M. Douze, and C. Schmid, "Packing Bag-of-Features," Proc. IEEE 12th Int'l Conf. Computer Vision, 2009.
[10] B. Kulis and K. Grauman, "Kernelized Locality-Sensitive Hashing for Scalable Image Search," Proc. IEEE 12th Int'l Conf. Computer Vision, 2009.
[11] N. Kumar, A.C. Berg, P.N. Belhumeur, and S.K. Nayar, "Attribute, and Simile Classifiers for Face Verification," Proc. IEEE 12th Int'l Conf. Computer Vision, 2009.
[12] P.-H. Lee, G.-S. Hsu, and Y.-P. Hung, "Face Verification, and Identification Using Facial Trait Code," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2009.
[13] Z. Lei, S. Li, R. Chu, and X. Zhu, "Face Recognition with Local Gabor Textons," Proc. Int'l Conf. Biometrics, pp. 49-57, 2007.
[14] L. Liang, R. Xiao, F. Wen, and J. Sun, "Face Alignment via Component-Based Discriminative Search," Proc. 10th European Conf. Computer Vision, 2008.
[15] D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 20, pp. 91-110, 2003.
[16] C.D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge Univ. Press, 2008.
[17] C.D. Manning, P. Raghavan, and H. Schütze, "Relevance Feedback and Query Expansion," Introduction to Information Retrieval, pp. 177-194, Cambridge Univ. Press, 2008.
[18] J. Matas, O. Chum, M. Urban, and T. Pajdla, "Robust Wide Baseline Stereo from Maximally Stable Extremal Regions," Proc. British Machine Vision Conf., 2002.
[19] K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. Van Gool, "A Comparison of Affine Region Detectors," Int'l J. Computer Vision, vol. 65, pp. 43-72, 2005.
[20] K. Mikolajczyk and C. Schmid, "A Performance Evaluation of Local Descriptors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, Oct. 2005.
[21] D. Nister and H. Stewenius, "Scalable Recognition with a Vocabulary Tree," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[22] T. Ojala, M. Pietikainen, and T. Maenpaa, "Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971-987, July 2002.
[23] J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object Retrieval with Large Vocabularies, and Fast Spatial Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[24] J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Lost in Quantization: Improving Particular Object Retrieval in Large Scale Image Databases," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[25] N. Pinto, J. Dicarlo, and D. Cox, "How Far Can You Get with a Modern Face Recognition Test Set Using Only Simple Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[26] S. Rudinac, M. Larson, and A. Hanjalic, "Exploiting Visual Reranking to Improve Pseudo-Relevance Feedback for Spoken-Content-Based Video Retrieval," Proc. Workshop Image Analysis for Multimedia Interactive Services, 2009.
[27] G. Salton, "Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer," Proc. Int'l Symp. Mobile Agents, 1989.
[28] J. Sivic and A. Zisserman, "Video Google: A Text Retrieval Approach to Object Matching in Videos," Proc. IEEE Ninth Int'l Conf. Computer Vision, Oct. 2003.
[29] Y. Taigman, L. Wolf, T. Hassner, and I. Tel-Aviv, "Multiple One-Shots for Utilizing Class Label Information," Proc. British Machine Vision Conf., 2009.
[30] X. Tan and B. Triggs, "Enhanced Local Texture Feature Sets for Face Recognition under Difficult Lighting Conditions," Proc. Third Int'l Conf. Analysis and Modeling of Faces and Gestures, pp. 168-182, 2007.
[31] P. Viola and M. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2001.
[32] S.A.J. Winder and M. Brown, "Learning Local Image Descriptors," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[33] L. Wiskott, J. Fellous, N. Kruger, and C. Von der Malsburg, "Face Recognition by Elastic Bunch Graph Matching," IEEE Trans. Pattern Analysis and Machine Intellignece, vol. 19, no. 7, pp. 775-779, July 1997.
[34] L. Wolf, T. Hassner, and Y. Taigman, "Descriptor Based Methods in the Wild," Proc. Faces in Real-Life Images Workshop European Conf. Computer Vision, 2008.
[35] J. Wright and G. Hua, "Implicit Elastic Matching with Random Projections for Pose-Variant Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[36] R. Yan, A. Hauptmann, and R. Jin, "Multimedia Search with Pseudo-Relevance Feedback," Proc. Int'l Conf. Image and Video Retrieval, 2003.
[37] R. Yan, A.G. Hauptmann, and R. Jin, "Negative Pseudo-Relevance Feedback in Content-Based Video Retrieval," Proc. ACM 11th Int'l Conf. Multimedia, pp. 343-346, 2003.
[38] L. Zhang, R. Chu, S. Xiang, S. Liao, and S. Li, "Face Detection Based on Multi-Block LBP Representation," Proc. Int'l Conf. Biometrics, pp. 11-18, 2007.
32 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool