This Article 
 Bibliographic References 
 Add to: 
Integrating Relevance Feedback Techniques for Image Retrieval Using Reinforcement Learning
October 2005 (vol. 27 no. 10)
pp. 1536-1551
Relevance feedback (RF) is an interactive process which refines the retrievals to a particular query by utilizing the user's feedback on previously retrieved results. Most researchers strive to develop new RF techniques and ignore the advantages of existing ones. In this paper, we propose an image relevance reinforcement learning (IRRL) model for integrating existing RF techniques in a content-based image retrieval system. Various integration schemes are presented and a long-term shared memory is used to exploit the retrieval experience from multiple users. Also, a concept digesting method is proposed to reduce the complexity of storage demand. The experimental results manifest that the integration of multiple RF approaches gives better retrieval performance than using one RF technique alone, and that the sharing of relevance knowledge between multiple query sessions significantly improves the performance. Further, the storage demand is significantly reduced by the concept digesting technique. This shows the scalability of the proposed model with the increasing-size of database.

[1] M. Flickner and H. Sawhney, “Query by Image and Video Content: The QBIC System,” Computer, vol. 28, no. 9, pp. 23-32, Sept. 1995.
[2] Pattern Recognition, J.C.M. Lee and A.K. Jain eds., special issue on image database, vol. 30, no. 4, 1997.
[3] Y. Rui, T.S. Huang, and S.F. Chang, “Image Retrieval: Current Techniques, Promising Directions, and Open Issues,” J. Visual Comm. and Image Representation, vol. 10, no. 1, pp. 39-62, 1999.
[4] A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-Based Image Retrieval at the End of the Early Years,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, Dec. 2000.
[5] B. Moghaddam, H. Biermann, and D. Margaritis, “Regions-of-Interest and Spatial Layout for Content-Based Image Retrieval,” Multimedia Tools and Applications, vol. 14, no. 2, pp. 201-210, 2001.
[6] J.J. Rocchio Jr., “Relevance Feedback in Information Retrieval,” The SMART System, G. Salton, ed., pp. 313-323, Prentice Hall, 1971.
[7] E. Ide and G. Salton, “Interactive Search Strategies and Dynamic File Organization in Information Retrieval,” The SMART System, G. Salton, ed., pp. 373-393, Prentice Hall, 1971.
[8] Y. Rui, T.S. Huang, and S. Mehrotra, “Relevance Feedback Techniques in Interactive Content-Based Image Retrieval,” Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases, pp. 25-36, 1998.
[9] Y. Rui, T.S. Huang, M. Ortega, and S. Mehrotra, “Relevance Feedback: A Power Tool for Interactive Content-Based Image Retrieval,” IEEE Trans. Circuits and Systems for Video Technology, vol. 8, no. 5, pp. 644-655, 1998.
[10] J. Peng, B. Bhanu, and S. Qing, “Probabilistic Feature Relevance Learning for Content-Based Image Retrieval,” Computer Vision and Image Understanding, vol. 75, nos. 1-2, pp. 150-164, 1999.
[11] S. Aksoy, R.M. Haralick, F.A. Cheikh, and M. Gabbouj, “Weighted Distance Approach to Relevance Feedback,” Proc. Int'l Conf. Pattern Recognition, pp. 812-815, 2000.
[12] I. Cox, M. Miller, T. Minka, S. Omohundro, and P. Yianilos, “Target Testing and the Pichunter Bayesian Multimedia Retrieval System,” Proc. Research and Technology Advances in Digital Libraries, pp. 66-75, 1996.
[13] C. Meilhac and C. Nastar, “Relevance Feedback and Category Search In Image Database,” Proc. Int'l Conf. Multimedia Computing and Systems, pp. 512-517, 1999.
[14] I. Cox, M. Miller, T. Minka, T. Papathomas, and P. Yianilos, “The Bayesian Image Retrieval System, Pichunter: Theory, Implementation, and Psychophysical Experiments,” IEEE Trans. Image Processing, vol. 9, no. 1, pp. 20-37, 2000.
[15] B. Bhanu and A. Dong, “Exploitation of Meta Knowledge for Learning Visual Concepts,” Proc. IEEE Workshop Content-Based Access of Image and Video Libraries, pp. 81-88, 2001.
[16] B. Bhanu and A. Dong, “Concept Learning with Fuzzy Clustering and Relevance Feedback,” Eng. Applications and Artificial Intelligence, vol. 15, pp. 123-138, 2002.
[17] P.Y. Yin, B. Bhanu, K.C. Chang, and A. Dong, “Improving Retrieval Performance by Long-Term Relevance Information,” Proc. Int'l Conf. Pattern Recognition, vol. 3, pp. 533-536, 2002.
[18] R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction. MIT Press, 1998.
[19] L.P. Kaelbling and A.W. Moore, “Reinforcement Learning: A Survey,” J. Artificial Intelligence Research, vol. 4, pp. 237-285, 1996.
[20] J. Peng and B. Bhanu, “Delayed Reinforcement Learning for Adaptive Image Segmentation and Feature Extraction,” IEEE Trans. System, Man, and Cybernetics— Part C, vol. 28, no. 3, pp. 482-488, 1998.
[21] J. Peng and B. Bhanu, “Closed-Loop Object Recognition Using Reinforcement Learning,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 2, pp. 139-154, Feb. 1998.
[22] D. Heesch and S. Ruger, “Relevance Feedback for Content-Based Image Retrieval: What Can Three Mouse Clicks Achieve,” Proc. European Conf. Information Retrieval Research, 2003.
[23] J. Lim, J. Wu, S. Singh, and D. Narasimhalu, “Learning Similarity Matching in Multimedia Content-Based Retrieval,” IEEE Trans. Knowledge and Data Eng., vol. 13, pp. 846-850, 2001.
[24] J. Smith, S. Srinivasan, A. Amir, S. Basu, G. Iyengar, C. Lin, M. Naphade, D. Ponceleon, and B. Tseng, “Integrating Features, Models, and Semantics for TREC Video Retrieval,” Proc. Text REtrieval Conf. (TREC 2001), pp. 240-249, 2001.
[25] S. Aksoy and R. Haralick, “Classification Framework for Content-Based Image Retrieval,” Proc. Int'l Conf. Pattern Recognition, pp. 240-249, 2002.
[26] C. Watkins and P. Dayan, “Q-Learning,” Machine Learning, vol. 8, pp. 279-292, 1992.
[27] Visualization and Intelligent Systems Lab (VISLab), Univ. of California, Riverside, http:/, 2005.
[28] K.R. Castleman, Digital Image Processing. Prentice-Hall, 1996.

Index Terms:
Index Terms- Content-based image retrieval, long-term learning, reinforcement learning, relevance feedback, short-term learning.
Peng-Yeng Yin, Bir Bhanu, Kuang-Cheng Chang, Anlei Dong, "Integrating Relevance Feedback Techniques for Image Retrieval Using Reinforcement Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1536-1551, Oct. 2005, doi:10.1109/TPAMI.2005.201
Usage of this product signifies your acceptance of the Terms of Use.