The Community for Technology Leaders
RSS Icon
Issue No.06 - June (2009 vol.31)
pp: 1087-1101
Marin Ferecatu , Institut Telecom, Telecom Paristech, Paris
Donald Geman , The Johns Hopkins University, Baltimore
Traditional image retrieval techniques usually require a first image to start a query. However, for large unstructured databases, it is not clear how to choose this query (the "page zero" problem). We propose a new statistical framework based on relevance feedback to locate an instance of a semantic category in an unstructured image database with no semantic annotation. A search session is initiated from a random sample of images. At each retrieval round the user is asked to select one image from among a set of displayed images - the one that is closest in his opinion to the target class. The matching is then "mental". Performance is measured by the number of iterations necessary to display an image which satisfies the user, at which point standard techniques can be employed to display other instances. Our core contribution is a Bayesian formulation which scales to large databases. The two key components are a response model which accounts for the user's subjective perception of similarity and a display algorithm which seeks to maximize the flow of information. Experiments with real users and two databases of 20,000 and 60,000 images demonstrate the efficiency of the search process.
Relevance Feedback, Image Retrieval, Page Zero Problem, Bayesian System, Statistical Learning, Mental Matching
Marin Ferecatu, Donald Geman, "A Statistical Framework for Image Category Search from a Mental Picture", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 6, pp. 1087-1101, June 2009, doi:10.1109/TPAMI.2008.259
[1] R. Datta, D. Joshi, J. Li, and J. Wang, “Image Retrieval: Ideas, Influences, and Trends of the New Age,” ACM Computing Surveys, vol. 40, no. 2, pp.5:1-60, 2008.
[2] M. Lew, N. Sebe, C. Djeraba, and R. Jain, “Content-Based Multimedia Information Retrieval: State-of-the-Art and Challenges,” ACM Trans. Multimedia Computing, Comm., and Applications, vol. 2, no. 1, pp.1-19, 2006.
[3] A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-Based Image Retrieval at the End of the Early Years.” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp.1349-1380, Dec. 2000.
[4] T. Gevers and A.W.M. Smeulders, “Content-Based Image Retrieval: An Overview,” Emerging Topics in Computer Vision, G.Medioni and S.B. Kang, eds., Prentice-Hall, 2004.
[5] X.S. Zhou and T.S. Huang, “Relevance Feedback for Image Retrieval: A Comprehensive Review,” Multimedia Systems, vol. 8, no. 6, pp.536-544, 2003.
[6] B.L. Saux and N. Boujemaa, “Image Database Clustering with SVM-Based Class Personalization,” Proc. SPIE Conf. Storage and Retrieval Methods and Applications for Multimedia, part of Electronic Imaging Symp., 2004.
[7] J. Fauqueur and N. Boujemaa, “Mental Image Search by Boolean Composition of Region Categories,” Multimedia Tools and Applications, vol. 31, no. 1, pp.95-117, 2006.
[8] I.J. Cox, M.L. Miller, T.P. Minka, T. Papathomas, and P.N. Yianilos, “The Bayesian Image Retrieval System, PicHunter: Theory, Implementation and Psychophysical Experiments,” IEEE Trans. Image Processing, vol. 9, no. 1, pp.20-37, Jan. 2000.
[9] Y. Fang and D. Geman, “Experiments in Mental Face Retrieval,” Proc. Audio- and Video-Based Biometric Person Authentication, pp.637-646, 2005.
[10] G. Carneiro, A.B. Chan, P.J. Moreno, and N. Vasconcelos, “Supervised Learning of Semantic Classes for Image Annotation and Retrieval,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 3, pp.394-410, Mar. 2006.
[11] J. Li and J.Z. Wang, “Real-Time Computerized Annotation of Pictures,” Proc. ACM Multimedia Conf., pp.911-920, 2006.
[12] M. Ferecatu and D. Geman, “Interactive Search for Image Categories by Mental Matching,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[13] V. Vapnik, Estimation of Dependencies Based on Empirical Data. Springer Verlag, 1982.
[14] B. Schölkopf and A. Smola, Learning with Kernels. MIT Press, 2002.
[15] S. Tong and E. Chang, “Support Vector Machine Active Learning for Image Retrieval,” Proc. Ninth ACM Int'l Conf. Multimedia, pp.107-118, 2001, .
[16] M. Ferecatu, N. Boujemaa, and M. Crucianu, “Semantic Interactive Image Retrieval Combining Visual and Conceptual Content Description,” ACM Multimedia Systems J., vol. 13, nos.5/6, pp.309-322, 2008.
[17] T. Huang, C. Dagli, S. Rajaram, E. Chang, M. Mandel, G. Poliner, and D. Ellis, “Active Learning for Interactive Multimedia Retrieval,” Proc. IEEE, vol. 96, no. 4, pp.648-667, 2008.
[18] K. Goh, E. Chang, and W. Lai, “Multimodal Concept Dependent Active Learning for Image Retrieval,” Proc. Ninth ACM Int'l Conf. Multimedia, 2004.
[19] X. He, W. Ma, and H. Zhang, “Learning An Image Manifold for Retrieval,” Proc. Ninth ACM Int'l Conf. Multimedia, 2004.
[20] H. Sahbi, P. Etyngier, J.-Y. Audibert, and R. Keriven, “Manifold Learning Using Robust Graph Laplacian for Interactive Image Retrieval,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[21] C. Hoi and M. Lyu, “A Novel Log-Based Relevance Feedback Technique in Content Based Image Retrieval,” Proc. Ninth ACM Int'l Conf. Multimedia, 2004.
[22] Z.-H. Zhou, K.-J. Chen, and H.-B. Dai, “Enhancing Relevance Feedback in Image Retrieval Using Unlabeled Data,” ACM Trans. Information Systems, vol. 24, no. 2, pp.219-244, 2006.
[23] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” Computer, vol. 28, no. 9, pp. 23-32, Sept. 1995.
[24] A. del Bimbo and P. Pala, “Visual Image Retrieval by Elastic Matching of User Sketches,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 2, pp.121-132, Feb. 1997.
[25] A. Chalechale, G. Naghdy, and A. Mertins, “Sketch-based Image Matching Using Angular Partitioning,” IEEE Trans. Systems, Man, and Cybernetics vol. 35, no. 1, pp.28-41, 2005.
[26] B. Ko and H. Byun, “Integrated Region-Based Image Retrieval Using Region's Spatial Relationships,” Proc. IEEE Int'l Conf. Pattern Recognition, 2002.
[27] G. Caenen and E.J. Pauwels, “Logistic Regression Model for Relevance Feedback in Content-Based Image Retrieval,” Proc. Storage and Retrieval for Media Databases, pp.49-58, 2001.
[28] N. Vasconcelos and A. Lippman, “A Probabilistic Architecture for Content-Based Image Retrieval,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000.
[29] N. Vasconcelos and A. Lippman, “Learning from User Feedback in Image Retrieval Systems,” Proc. Conf. Advances in Neural Information Processing Systems, 2000.
[30] Z. Su, H.-J. Zhang, S. Li, and S. Andma, “Relevance Feedback in Content-Based Image Retrieval: Bayesian Framework, Feature Subspaces, and Progressive Learning,” IEEE Trans. Image Processing, vol. 12, no. 8, pp.924-937, 2003.
[31] K. Mikolajczyk and C. Schmid, “A Performance Evaluation of Local Descriptors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp.1615-1630, Oct. 2005.
[32] M. Ferecatu, “Image Retrieval with Active Relevance Feedback Using Both Visual and Keyword-Based Descriptors,” PhD dissertation, INRIA—Univ. of Versailles Saint Quentin-en-Yvelines, France, 2005.
[33] C. Vertan and N. Boujemaa, “Upgrading Color Distributions for Image Retrieval: Can We Do Better?” Proc. Int'l Conf. Visual Information Systems, Nov. 2000.
[34] Introduction to MPEG-7: Multimedia Content Description Interface, B.Manjunath, P. Salembier, and T. Sikora, eds. Wiley, 2002.
[35] I. Jolliffe, Principal Component Analysis. Springer-Verlag, 2002.
[36] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification. Wiley Interscience, 2001.
[37] H. Frigui and R. Krishnapuram, “A Robust Competitive Clustering Algorithm with Applications in Computer Vision,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp.450-465, May 1999.
[38] L. Heyer, S. Kruglyak, and S. Yooseph, “Exploring Expression Data: Identification and Analysis of Coexpressed Genes,” Genome, vol. 9, no. 11, pp.1106-1115, 1999.
22 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool