The Community for Technology Leaders
RSS Icon
Issue No.11 - November (2008 vol.30)
pp: 1902-1912
Sally A. Goldman , Washington University, St. Louis
Hui Zhang , Washington University, St. Louis
Sharath R. Cholleti , Washington University, St. Louis
Jason E. Fritts , St. Louis University, St. Louis
We define localized content-based image retrieval as a CBIR task where the user is only interested in a portion of the image, and the rest of the image is irrelevant. In this paper we present a localized CBIR system, Accio, that uses labeled images in conjunction with a multiple-instance learning algorithm to first identify the desired object and weight the features accordingly, and then to rank images in the database using a similarity measure that is based upon only the relevant portions of the image. A challenge for localized CBIR is how to represent the image to capture the content. We present and compare two novel image representations, which extend traditional segmentation-based and salient point-based techniques respectively, to capture content in a localized CBIR setting.
Information Search and Retrieval, Relevance feedback, Machine learning
Sally A. Goldman, Hui Zhang, Sharath R. Cholleti, Jason E. Fritts, "Localized Content-Based Image Retrieval", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 11, pp. 1902-1912, November 2008, doi:10.1109/TPAMI.2008.112
[1] S. Andrews, T. Hofmann, and I. Tsochantaridis, “Multiple Instance Learning with Generalized Support Vector Machines,” Artificial Intelligence, pp. 943-944, 2002.
[2] K. Barnard, P. Duygulu, N. de Freitas, and D. Forsyth, “Matching Words and Pictures,” J. Machine Learning Research, vol. 3, pp. 1107-1135, 2003.
[3] J. Bi, Y. Chen, and J. Wang, “A Sparse Support Vector Machine Approach to Region-Based Image Categorization,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '05), pp. 1121-1128, 2005.
[4] Y. Chen and J. Wang, “Image Categorization by Learning and Reasoning with Regions,” J. Machine Learning Research, pp. 913-939, 2004.
[5] I. Cox, M. Miller, S. Omohundro, and P. Yianilos, “PicHunter: Bayesian Relevance Feedback,” Proc. Int'l Conf. Pattern Recognition (ICPR '96), pp. 361-369, 1996.
[6] A. Dempster, N. Laird, and D. Rubin, “Maximum Likelihood from Incomplete Data via the EM Algorithm,” J. Royal Statistics Soc., vol. 39, pp. 1-38, 1977.
[7] T. Dietterich, R. Lathrop, and T. Lozano-Pérez, “Solving the Multiple-Instance Problem with Axis-Parallel Rectangles,” Artificial Intelligence, vol. 89, nos. 1-2, pp. 31-37, 1997.
[8] P. Duygulu, K. Barnard, N. de Freitas, and D. Forsyth, “Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary,” Proc. Seventh European Conf. Computer Vision (ECCV '02), pp. IV:97-IV:12, 2002.
[9] M. Fischler and R. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Comm. ACM, vol. 24, pp. 381-395, 1981.
[10] D. Forsyth and J. Ponce, Computer Vision: A Modern Approach. Prentice Hall, 2003.
[11] T. Gevers and A. Smeulders, “Image Search Engines: An Overview,” Emerging Topics in Computer Vision, G. Medioni and S.B.Kang, eds., Prentice Hall, 2004.
[12] J. Hanley and B. McNeil, “The Meaning and Use of the Area under a Receiver Operating Characteristic (ROC) Curve,” Radiology, vol. 143, no. 1, pp. 29-36, 1982.
[13] X. Huang, S.-C. Chen, M.-L. Shyu, and C. Zhang, “User Concept Pattern Discovery Using Relevance Feedback and Multiple Instance Learning for Content-Based Image Retrieval,” Proc. Eighth Int'l Conf. Knowledge Discovery and Data Mining (KDD '02), pp. 100-108, 2002.
[14] J. Wang, J. Li, and G. Wiederhold, “IRM: Integrated Region Matching for Image Retrieval,” Proc. Eighth ACM Int'l Conf. Multimedia, pp. 147-156, 2000.
[15] Y. Ke and R. Sukthankar, “PCA-SIFT: A More Distinctive Representation for Local Image Descriptors,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '04), vol. 2, pp. 506-513, 2004.
[16] L. Ledwich and S. Williams, “Reduced Sift Features for Image Retrieval and Indoor Localisation,” Proc. Australasian Conf. Robotics and Automation (ACRA), 2004.
[17] H.-K. Lee and Y.-S. Ho, “A Region-Based Image Retrieval System Using Salient Point Extraction and Image Segmentation,” Proc. Third IEEE Pacific Rim Conf. Multimedia (PCM '02), pp. 209-216, 2002.
[18] F. Long, H. Zhang, and D. Feng, “Fundamentals of Content-Based Image Retrieval,” Multimedia Information Retrieval and Management-Technological Fundamentals and Applications, D. Feng, W. Siu, and H. Zhang, eds., Springer, 2003.
[19] E. Loupias, Salient Points Detection Using Wavelet Transform, demo.html, 2008.
[20] D.G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[21] O. Maron and T. Lozano-Pérez, “A Framework for Multiple-Instance Learning,” Proc. Conf. Advances in Neural Information Processing Systems (NIPS '98), pp. 570-576, 1998.
[22] O. Maron and A. Ratan, “Multiple-Instance Learning for Natural Scene Classification,” Proc. 15th Int'l Conf. Machine Learning (ICML '98), pp. 341-349, 1998.
[23] K. Mikolajczyk and C. Schmid, “A Performance Evaluation of Local Descriptors,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '03), pp. 257-264, 2003.
[24] R. Rahmani, S. Goldman, H. Zhang, J. Krettek, and J. Fritts, “Localized Content-Based Image Retrieval,” Proc. Seventh ACM SIGMM Int'l Workshop Multimedia Information Retrieval (MIR '05), pp. 227-236, 2005.
[25] Y. Rui, T. Huang, M. Ortega, and S. Mehrotra, “Relevance Feedback: A Power Tool in Interactive Content-Based Image Retrieval,” IEEE Trans. Circuits and Systems for Video Technology, vol. 8, no. 5, pp. 644-655, 1998.
[26] C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of Interest Point Detectors,” Int'l J. Computer Vision, vol. 37, no. 2, pp. 151-172, 2000.
[27] Q. Tian, N. Sebe, M.S. Lew, E. Loupias, and T.S. Huang, “Image Retrieval Using Wavelet-Based Salient Points,” J. Electronic Imaging, special issue on Storage and Retrieval of Digital Media, vol. 10, no. 4, pp. 849-935, 2001.
[28] Q. Tian, Y. Wu, and T. Huang, “Combine User Defined Region-of-Interest and Spatial Layout for Image Retrieval,” Proc. IEEE Conf. Image Processing (ICIP '00), vol. 3, pp. 746-749, 2000.
[29] P. Torr and A. Zisserman, “Mlesac: A New Robust Estimator with Application to Estimating Image Geometry,” Computer Vision and Image Understanding, vol. 78, no. 1, pp. 138-156, 2000.
[30] J. Wang, J. Li, and G. Wiederhold, “SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture Libraries,” IEEE Trans. Pattern Analysis and Machine Intelligence, pp. 947-963, 2001.
[31] J. Wang, H. Zha, and R. Cipolla, “Combining Interest Points and Edges for Content-Based Image Retrieval,” Proc. IEEE Int'l Conf. Image Processing (ICIP '05), pp. 1256-1259, 2005.
[32] C. Wolf, J.-M. Jolion, W. Kropatsch, and H. Bischof, “Content Based Image Retrieval Using Interest Points and Texture Features,” Proc. 15th IEEE Int'l Conf. Pattern Recognition (ICPR '00), vol. 4, p. 4234, 2000.
[33] C. Yang and T. Lozano-Pérez, “Image Database Retrieval with Multiple Instance Techniques,” Proc. 16th Int'l Conf. Data Eng. (ICDE '00), pp. 233-243, 2000.
[34] H. Zhang, R. Rahmani, S. Cholleti, and S. Goldman, “Local Image Representations Using Pruned Salient Points with Applications to CBIR,” Proc. 14th ACM Int'l Conf. Multimedia, pp. 287-296, 2006.
[35] H. Zhang, J. Fritts, and S. Goldman, “An Improved Fine-Grain Hierarchical Method of Image Segmentation,” technical report, Washington Univ., 2005.
[36] H. Zhang, J. Fritts, and S. Goldman, “A Fast Texture Feature Extraction Method for Region-Based Image Segmentation,” Proc. IS&T/SPIE's 16th Ann. Symp. Image and Video Comm. and Processing, pp. 957-968, 2005.
[37] Q. Zhang and S. Goldman, “EM-DD: An Improved Multiple-Instance Learning Technique,” Neural Information Processing Systems, pp. 1073-1080, 2001.
[38] Q. Zhang, S. Goldman, W. Yu, and J. Fritts, “Content-Based Image Retrieval Using Multiple Instance Learning,” Proc. 19th Int'l Conf. Machine Learning (ICML '02), pp. 682-689, 2002.
[39] Z. Zhou and M. Zhang, “Ensembles of Multi-Instance Learners,” Proc. 14th European Conf. Machine Learning (ECML '03), pp. 492-502, 2003.
12 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool