This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Content-Based Image Retrieval Based on a Fuzzy Approach
October 2004 (vol. 16 no. 10)
pp. 1185-1199
A typical content-based image retrieval (CBIR) system would need to handle the vagueness in the user queries as well as the inherent uncertainty in image representation, similarity measure, and relevance feedback. In this paper, we discuss how fuzzy set theory can be effectively used for this purpose and describe an image retrieval system called FIRST (Fuzzy Image Retrieval SysTem) which incorporates many of these ideas. FIRST can handle exemplar-based, graphical-sketch-based, as well as linguistic queries involving region labels, attributes, and spatial relations. FIRST uses Fuzzy Attributed Relational Graphs (FARGs) to represent images, where each node in the graph represents an image region and each edge represents a relation between two regions. The given query is converted to a FARG, and a low-complexity fuzzy graph matching algorithm is used to compare the query graph with the FARGs in the database. The use of an indexing scheme based on a leader clustering algorithm avoids an exhaustive search of the FARG database. We quantify the retrieval performance of the system in terms of several standard measures.

[1] Y. Rui, T.S. Huang, and S.F. Chang, Image Retrieval: Current Techniques, Promising Directions and Open Issues J. Visual Comm. and Image Representation, vol. 10, no. 4, pp. 39-62, Apr. 1999.
[2] R.C. Jain, Special Issue on Visual Information Management Comm. ACM, vol. 40, no. 12, pp. 30-32, Dec. 1997.
[3] V.N. Gudivada and V. Raghavan, Special Issue on Content-Based Image Retrieval Systems Computer, vol. 28, no. 9, Sept. 1995.
[4] A. Pentland and R. Picard, Special Issue on Digital Libraries IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 783-789, Aug. 1996.
[5] A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, Content-Based Image Retrieval at the End of the Early Years IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, Dec. 2000.
[6] S. Santini and R. Jain, “Similarity Measures,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 9, pp. 871-883, Sept. 1999.
[7] V.N. Gudivada and V.V. Raghavan, Design and Evaluation of Algorithms for Image Retrieval by Spatial Similarity ACM Trans. Information Systems, vol. 13, no. 2, pp. 115-144, Apr. 1995.
[8] T. Hermes, C. Klauck, J. Kreys, and J. Zhang, Image Retrieval for Information Systems Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases, pp. 394-407, Feb. 1995.
[9] D.A. Forsyth, J. Malik, T.K. Leung, C. Bregler, C. Carson, H. Greenspan, and M.M. Fleck, Finding Pictures of Objects in Large Collections of Images Proc. Int'l Workshop Object Recognition for Computer Vision, pp. 335-360, Apr. 1996.
[10] J.R. Smith and S.F. Chang, VisualSEEk: A Fully Automated Content-Based Image Query System Proc. ACM Int'l Conf. Multimedia, pp. 87-98, Nov. 1996.
[11] A. Del Bimbo and E. Vicario, “Using Weighted Spatial Relationships in Retrieval by Visual Contents,” IEEE Workshop Content-Based Access of Image and Video Databases, June 1998.
[12] W.Y. Ma and B.S. Manjunath, “NETRA: A Toolbox for Navigating Large Image Databases,” Proc. IEEE Int'l Conf. Image Processing, 1997.
[13] E.G.M. Petrakis and C. Faloutsos, “Similarity Searching in Medical Image Databases,” IEEE Trans. Knowledge and Data Eng., vol. 9, no. 3, pp. 435-447, May/June 1997.
[14] S. Medasani and R. Krishnapuram, A Fuzzy Approach to Content-Based Image Retrieval Proc. IEEE Int'l Conf. Fuzzy Systems, vol. 3, pp. 1251-1260, 1999.
[15] J. Freeman, The Modeling of Spatial Relations Computer Graphics and Image Processing, vol. 4, pp. 156-171, 1975.
[16] R. Krishnapuram, J.M. Keller, and Y. Ma, “Quantitative Analysis of Properties and Spatial Relations of Fuzzy Image Regions,” IEEE Trans. Fuzzy Systems, vol. 1, no. 3, pp. 222-233, 1993.
[17] K.P. Chan and Y.S. Cheung, Fuzzy-Attribute Graph with Application to Chinese Character Recognition IEEE Trans. Systems, Man, and Cybernetics, vol. 22, no. 1, pp. 153-160, Jan./Feb. 1992.
[18] S. Gold and A. Rangarajan, “A Graduated Assignment Algorithm for Graph Matching,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 4, pp. 377-388, Apr. 1996.
[19] M.J. Swain and D.H. Ballard, Color Indexing Int'l J. Computer Vision, vol. 7, no. 1, pp. 11-32, 1991.
[20] M. Stricker and M. Orengo, Similarity of Color Images Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases III, W.R. Niblack and R.C. Jain, eds., pp. 381-392, 1995.
[21] C. Carson, S. Belongie, H. Greenspan, and J. Malik, “Region-Based Image Querying,” Proc. Int'l Workshop Content-Based Access of Image and Video libraries, 1997.
[22] J.R. Smith and S.F. Chang, Tools and Techniques for Color Image Retrieval Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases IV, pp. 426-437, 1995.
[23] M.R. Haralick, K. Shanmugam, and I. Dinstein, Texture Features for Image Classification IEEE Trans. Systems, Man, and Cybernetics, vol. 3, no. 6, pp. 610-621, 1973.
[24] H. Tamura, S. Morei, and T. Yamawaki, Texture Features Corresponding to Visual Perception IEEE Trans. Systems, Man, and Cybernetics, vol. 8, no. 6, pp. 460-473, 1978.
[25] B.S. Manjunath and W.Y. Ma, “Texture Features for Browsing and Retrieval of Image Data,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 837-842, Aug. 1996
[26] F. Liu and R.W. Picard, “Periodicity, Directionality, and Randomness: Wold Features for Image Modelling and Retrieval,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 7, pp. 722-733, July 1996.
[27] B.S. Manjunath and W.Y. Ma, A Pattern Thesaurus for Browsing Large Aerial Photographs Technical Report 96-10, Univ. of California at Santa Barbara, 1996.
[28] W. Niblack, The GBIC Project: Querying Images by Content Color Texture and Shape Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases, pp. 173-197, 1993.
[29] R. Mehrotra and J.E. Gary, “Similar-Shape Retrieval in Shape Data Management,” Computer, vol. 28, no. 9, pp. 57-62, Sept. 1995.
[30] A. del Bimbo and P. Pala, “Visual Image Retrieval by Elastic Matching of User Sketches,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 2, pp. 121-132, Feb. 1997.
[31] A. Jain and A. Vailaya, Image Retrieval Using Color and Shape Pattern Recognition, vol. 29, no. 8, pp. 1233-1244, 1996.
[32] B. Kimia, J. Chan, D. Bertrand, S. Coe, Z. Roadhouse, and H. Tek, A Shock-Based Approach for Indexing of Image Databases Using Shape Proc. SPIE's Multimedia Storage and Archiving Systems II, pp. 288-302, Nov. 1997.
[33] M.A. Eshera and K.S. Fu, An Image Understanding System Using Attributed Symbolic Representation and Inexact Graph-Matching IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, pp. 604-619, Sept. 1986.
[34] D. Eager and J. Zahorjan, "Enhanced run-time support for shared memory parallel computing," ACM Trans. Computer Systems, vol. 11, pp. 1-32, Feb. 1993.
[35] K. Miyajima and A. Ralescu, Analysis of Spatial Relations between 2D Segmented Regions Proc. European Congress Fuzzy and Intelligent Technologies, pp. 48-54, 1993.
[36] J.M. Keller and X. Wang, “Learning Spatial Relationships in Computer Vision,” Proc. IEEE Fifth Int'l Conf. Fuzzy Systems, 1996.
[37] I. Bloch, Fuzzy Relative Position between Objects in Image Processing: A Morphological Approach IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 7, pp. 657-664, July 1999.
[38] P. Matsakis and L. Wendling, A New Way to Represent Relative Position between Areal Objects IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 7, pp. 634-643, July 1999.
[39] G. Salton, Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison Wesley, 1989.
[40] C.T. Meadow, R.R. Boyce, and D.H. Kraft, Text Information Retrieval Systems, second ed. Academic Press, 2000.
[41] D.H. Kraft and A. Bookstein, Evaluation of Information Retrieval Systems: A Decision Theory Approach J. Am. Soc. Information Science, vol. 29, pp. 31-40, 1978.
[42] H. Dyckhoff and W. Pedrycz, Generalized Mean as a Model of Compensation Connectives Fuzzy Sets and Systems, vol. 14, pp. 143-154, 1984.
[43] D. Dubois and H. Prade, A Review of Fuzzy Set Aggregation Connectives Information Sciences, vol. 36, nos. 1/2, pp. 85-121, 1985.
[44] M. Mizumoto, Pictorial Representations of Fuzzy Connectives I: Cases of T-Norms, T-Conorms, and Averaging Operators Fuzzy Sets and Systems, vol. 31, pp. 217-242, 1989.
[45] R. Krishnapuram and J. Lee, Fuzzy Connective-Based Hierarchical Networks for Decision Making Fuzzy Sets and Systems, vol. 46, no. 1, pp. 11-27, Feb. 1992.
[46] Y. Rui, T.S. Huang, M. Ortega, and S. Mehrotra, “Relevance Feednack: A Power Tool for Interactive Conten-Based Image Retrieval,” IEEE Trans. Circuits, and Video Technology, Sept. 1998.
[47] B. Bhanu, J. Peng, and S. Quing, Learning Feature Relevance and Similarity Metrics in Image Databases Proc. IEEE Workshop Content-Based Access of Image and Video Libraries, pp. 14-18, 1998.
[48] M. Sugeno, Fuzzy Measures and Fuzzy Integrals: A Survey Fuzzy Automata and Decision Process, M.M. Gupta et al., eds., pp. 89-102, 1977.
[49] M. Grabisch, On Equivalence Classes of Fuzzy Connectives: The Case of Fuzzy Integrals IEEE Trans. Fuzzy Systems, vol. 8, no. 1, pp. 96-109, 1995.
[50] R.R. Yager, "On Ordered Weighted Averaging Aggregation Operators in Multi-Criteria Decision Making," IEEE Trans. Systems, Man, and Cybernetics, vol. 18, 1988, pp. 183-190.
[51] H. Frigui, Adaptive Image Retrieval Using the Fuzzy Integral Proc. North Am. Fuzzy Information Processing Soc. Conf., pp. 575-578, 1999.
[52] H. Samet, The Design and Analysis of Spatial Data Structures. Addison Wesley, 1990.
[53] D.A. White and R. Jain, Similarity Indexing: Algorithms and Performance Proc. SPIE Storage and Retrieval for Image and Video Databases IV, vol. 2670, pp. 62-73, Feb. 1996.
[54] R. Kurniawati, J.S. Jin, and J.A. Shepherd, Techniques for Supporting Efficient Content-Based Retrieval in Multimedia Databases The Australian Computer J., vol. 29, no. 4, pp. 122-130, 1997.
[55] C. Faloutsos, Searching Multimedia Databases by Content. Kluwer Academic, 1996.
[56] K. Fukunaga and P.M. Narendra, A Branch and Bound Algorithm for Computing k-Nearest Neighbors IEEE Trans. Computers, vol. 24, no. 7, pp. 750-753, July 1975.
[57] S. Medasani and R. Krishnapuram, Categorization of Image Databases for Efficient Retrieval Using Robust Mixture Decomposition Proc. IEEE Workshop Content-Based Access of Image and Video Lib, pp. 50-54, June 1998.
[58] H.J. Zhang and D. Zhong, A Scheme for Visual Feature Based Image Retrieval Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases III, pp. 36-46, 1995.
[59] Y. Rui, T.S. Huang, S. Mehrotra, and M. Ortega, A Relevance Feedback Architecture for Content-Based Multimedia Information Retrieval System Proc. IEEE Workshop Content-Based Access of Images and Video Libraries, pp. 82-89, 1997.
[60] C. Meilhac and C. Nastar, “Relevance Feedback and Category Search in Image Databases,” Proc. Int'l Conf. Multimedia Computing and Systems, pp. 512-517, 1999.
[61] Y. Rui, T.S. Huang, and S. Mehrotra, Relevance Feedback Techniques in Interactive Content-Based Image Retrieval Proc. IS&T and SPIE Conf. Storage and Retrieval of Image and Video Databases, pp. 25-36, 1998.
[62] M.E.J. Wood, N.W. Campbell, and B.T. Thomas, Iterative Refinement by Relevance Feedback in Content-Based Digital Image Retrieval Proc. Sixth ACM Int'l Conf. Multimedia, pp. 13-20, Sept. 1998.
[63] Y. Choi, D. Kim, and R. Krishnapuram, Relevance Feedback for Content-Based Image Retrieval Using the Choquet Integral Proc. IEEE Int'l Conf. Multimedia and Expo, vol. 2, pp. 1207-1210, 2000.
[64] L. Shapiro and R.M. Haralick, A Metric for Comparing Relational Descriptions IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, pp. 90-94, 1985.
[65] R. Krishnapuram and R. Medasani, A Fuzzy Approach to Graph Matching Proc. IFSA Congress Conf., pp. 1029-1033, Aug. 1999.
[66] S. Medasani, R. Krishnapuram, and Y. Choi, Graph Matching by Relaxation of Fuzzy Assignments IEEE Trans. Fuzzy Systems, vol. 9, no. 1, pp. 173-182, 2001.
[67] M.P. Windham, Numerical Classification of Proximity Data with Assignment Measure J. Classification, vol. 2, pp. 157-172, 1985.
[68] R. Sinkhorn, A Relationship between Arbitrary Positive Matrices and Doubly Stochastic Matrices Annals of Math. Statistics, vol. 35, pp. 876-879, 1964.
[69] J. Hartigan, Clustering Algorithms. Wiley, 1975.
[70] D.H. Ballard and C.M. Brown, Computer Vision. Prentice Hall, 1982.

Index Terms:
Content-based image retrieval, fuzzy graph models, graph matching, graph clustering, indexing.
Citation:
Raghu Krishnapuram, Swarup Medasani, Sung-Hwan Jung, Young-Sik Choi, Rajesh Balasubramaniam, "Content-Based Image Retrieval Based on a Fuzzy Approach," IEEE Transactions on Knowledge and Data Engineering, vol. 16, no. 10, pp. 1185-1199, Oct. 2004, doi:10.1109/TKDE.2004.53
Usage of this product signifies your acceptance of the Terms of Use.