This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
SemQuery: Semantic Clustering and Querying on Heterogeneous Features for Visual Data
September/October 2002 (vol. 14 no. 5)
pp. 988-1002

Abstract—The effectiveness of the content-based image retrieval can be enhanced using heterogeneous features embedded in the images. However, since the features in texture, color, and shape are generated using different computation methods and thus may require different similarity measurements, the integration of the retrievals on heterogeneous features is a nontrivial task. In this paper, we present a semantics-based clustering and indexing approach, termed SemQuery, to support visual queries on heterogeneous features of images. Using this approach, the database images are classified based on their heterogeneous features. Each semantic image cluster contains a set of subclusters that are represented by the heterogeneous features that the images contain. An image is included in a semantic cluster if it falls within the scope of all the heterogeneous clusters of the semantic cluster. We also design a neural network model to merge the results of basic queries on individual features. A query processing strategy is then presented to support visual queries on heterogeneous features. An experimental analysis is conducted and presented to demonstrate the effectiveness and efficiency of the proposed approach.

[1] N. Ahuja and A. Rosenfeld, “Mosaic Models for Texture,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 3, no. 1, pp. 1-11, 1981.
[2] J.R. Bach, C. Fuller, A. Gupta, A. Hampapur, B. Horowitz, R. Jain, and C.F. Shu, “The Virage Image Search Engine: An Open Framework for Image Management,” Proc. SPIE, Storage and Retrieval for Still Image and Video Databases IV, pp. 76-87, Feb. 1996.
[3] J.R. Bach, S. Paul, and R. Jain, “A Visual Information Management System for the Interactive Retrieval of Faces,” IEEE Trans. Knowledge and Data Eng., vol. 5, no. 4, pp. 619-628, 1993.
[4] N. Beckmann, H.-P. Kriegel, R. Schneider, and B. Seeger, “The R*-Tree: An Efficient and Robust Access Method for Points and Rectangles,” Proc. ACM SIGMOD Conf. Management of Data, 1990.
[5] G.R. Cross and A.K. Jain, “Markov Random Field Texture Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 5, no. 1, pp. 25-39, 1983.
[6] G. Cybenko, “Continous Valued Neural Netwrks with Two Hidden Layers are Sufficient,” technical report, Dept. of Computer Science, Tufts Univ., Medford, Mass., 1988.
[7] G. Cybenko, “Approximation by Superimposing of a Sigmoidal Function,” Math. of Control, Signals, and Systems, vol. 2, pp. 303-314, 1989.
[8] E.R. Dougherty and J.B. Pelz, “Texture-Based Segmentation by Morphological Granulometrics,” Advanced Printing of Paper Summaries, Electronic Imaging '89, vol. 1, pp. 408-414, Boston, Oct. 1989.
[9] M. Ester, H. Kriegel, J. Sander, and X. Xu, “A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise,” Proc. Second Int'l Conf. KDD, pp. 226-231, 1996.
[10] R. Fagin, “Fuzzy Queries in Multimedia Database Systems,” Proc. 1998 ACM SIGACT-SIGMOD-SIGART Symp. Principles of Database Systems, 1998.
[11] R. Fagin, “Combining Fuzzy Information from Multiple Systems,” J. Computer Systems Science, vol. 58, no. 1, pp. 83-99, 1999.
[12] C. Faloutsos, R. Barber, M. Flicker, J. Hafner, W. Niblack, and W. Equitz, "Efficient and effective querying by image content," J. Intell. Information Systems," vol. 3, pp. 231-262, 1994.
[13] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” IEEE Computer, 1995.
[14] B. Furht, S.W. Smoliar, and H.J. Zhang, Video and Image Processing in Multimedia Systems. Boston: Kluwer Academic, 1996.
[15] A.D. Gordon, Classification Methods for the Exploratory Analysis of Multivariate Data. Chapman and Hall, 1981.
[16] Computer, special issue on content-based image retrieval systems, vol. 28, no. 9, V.N. Gudivada and V.V. Raghavan, eds., Sept. 1995.
[17] A. Guttman, “R-Trees: A Dynamic Index Structure for Spatial Searching,” Proc. ACM SIGMOD Conf. Management of Data, 1984.
[18] K. Hirata and T. Kato, “Rough Sketch-Based Image Information Retrieval,” NEC Research&Development, vol. 34, no. 2, pp. 263-273, 1993.
[19] B.K. Horn, Robot Vision. Cambridge, Mass.: MIT Press, 1986.
[20] K. Hornik, M. Stinchcombe, and H. White, “Multilayer Feedforward Networks are Universal Approximations,” Neural Networks, vol. 2, pp. 359-366, 1989.
[21] R.W. Fries, J.W. Modestino, and A.L. Vickers, “Texture Discrimination Based upon an Assumed Stochastic Texture Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 3, no. 5, pp. 557-580, 1981.
[22] L. Kaufman and P.J. Rousseeuw, Finding Groups in Data: An Introduction to Cluster Analysis. John Wiley&Sons, 1990.
[23] F. Korn, N. Sidiropoulos, C. Faloutsos, E. Siegel, and Z. Protopapas, “Fast Nearest-Neighbor Search in Medical Image Databases,” Proc. Conf. Very Large Data Bases (VLDB '96), Sept. 1996.
[24] F. Liu and R. Picard, “Periodicity, Directionality, and Randomness: Wold Features for Image Modeling and Retrieval,” Technical Report 320, MIT Media Laboratory, Perceptual Computing, 1996.
[25] D.G. Lowe, Perceptual Organization and Visual Recognition. Boston: Kluwer Academic, 1985.
[26] W.Y. Ma and B.S. Manjunath, “NETRA: A Toolbox for Navigating Large Image Databases,” Proc. IEEE Int'l Conf. Image Processing, 1997.
[27] B.B. Mandelbrot, Fractals—Form, Chance, Dimension. San Fransisco: W.H. Freeman, 1977.
[28] B.S. Manjunath and W.Y. Ma, “Texture Features for Browsing and Retrieval of Image Data,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 837-842, Aug. 1996
[29] R. Mehrotra and J.E. Gary, “Similar-Shape Retrieval in Shape Data Management,” Computer, vol. 28, no. 9, pp. 57-62, Sept. 1995.
[30] T.P. Minka and R.W. Picard, “Interactive Learning Using a‘Society of Models,“ Technical Report 349, MIT Media Laboratory, Perceptual Computing Section, 1995.
[31] T. Minka, “An Image Database Browser that Learns from User Interaction,” master's thesis, MIT, 1996.
[32] F. Mokhtarian, S. Abbasi, and J. Kittler, “Efficient and Robust Retrieval by Shape Content through Curvature Scale Space,” Proc. Int'l Workshop Image Databases and MultiMedia Search, pp. 35-42, 1996.
[33] F. Mokhtarian, S. Abbasi, and J. Kittler, “Robust and Efficient Shape Indexing through Curvature Scale Space,” Proc. British Machine Vision Conf., pp. 53-62, 1996.
[34] R.T. Ng and J. Han, "Efficient and Effective Clustering Methods for Spatial Data Mining," Proc. 20th Int'l Conf. Very Large Databases, Morgan Kaufmann, 1994, pp. 144-155.
[35] G. Pass, R. Zabih, and J. Miller, “Comparing Images Using Color Coherence Vectors,” Proc. ACM Multimedia '96, pp. 65-73, 1996.
[36] E.J. Pauwels, P. Fiddelaers, and L. Van Gool, “DOG-Based Unsupervized Clustering for CBIR,” Proc. Second Int'l Conf. Visual Information Systems, pp. 13-20, Dec. 1997.
[37] A. Pentland, R. Picard, and S. Sclaroff, “Photobook: Tools for Content-Based Manipulation of Image Databases,” Proc. SPIE Conf. Storage and Retrieval of Image and Video Databases II, pp. 34-47, 1994.
[38] R. Picard, “A Society of Models for Video and Image Libraries,” Technical Report 360, MIT Media Laboratory, Perceptual Computing, 1996.
[39] M. Safar, C. Shahabi, and X. Sun, “Image Retrieval by Shape: A Comparative Study,” Proc. IEEE Int'l Conf. Multimedia and Exposition (ICME), 2000.
[40] C. Shahabi and M. Safar, “Efficient Retrieval and Spatial Querying of 2D Objects,” Proc. IEEE Int'l Conf. Multimedia Computing and Systems (ICMCS), pp. 611-617, June 1999.
[41] G. Sheikholeslami, W. Chang, and A. Zhang, “Semantic Clustering and Querying on Heterogeneous Features for Visual Data,” Proc. Sixth ACM Int'l Multimedia Conf. (ACM Multimedia '98), pp. 3-12, Sept. 1998.
[42] G. Sheikholeslami, S. Chatterjee, and A. Zhang, WaveCluster: A Multi-Resolution Clustering Approach for Very Large Spatial Databases Proc. Very Large Date Bases Conf., pp. 428-439, Aug. 1998.
[43] G. Sheikholeslami, S. Chatterjee, and A. Zhang, “NeuroMerge: An Approach for Merging Heterogeneous Features in Content-Based Image Retrieval Systems,” Proc. 1998 Int'l Workshop Multi-Media Database Management Systems, pp. 106-113, Aug. 1998.
[44] G. Sheikholeslami, S. Chatterjee, and A. Zhang, “WaveCluster: A Wavelet-Based Clustering Approach for Multidimensional Data in Very Large Databases,” The VLDB J., vol. 8, no. 4, pp. 289-304, Feb. 2000.
[45] G. Sheikholeslami and A. Zhang, “An Approach to Clustering Large Visual Databases Using Wavelet Transform,” Proc. SPIE Conf. Visual Data Exploration and Analysis IV, pp. 322-333, Feb. 1997.
[46] G. Sheikholeslami, A. Zhang, and L. Bian, "Geographical Data Classification and Retrieval," Proc. 5th ACM Int'l Workshop on Geographic Information Systems, ACM Press, New York, Nov. 1997, pp. 58-61.
[47] G. Sheikholeslami, A. Zhang, and L. Bian, “A Multi-Resolution Content-Based Retrieval System for Geographic Images,” GoeInformatica, An Int'l J. Advances of Computer Science for Geographic Information Systems, vol. 3, no. 2, pp. 109-139, June 1999.
[48] J.R. Smith and S. Chang, “Transform Features For Texture Classification and Discrimination in Large Image Databases,” Proc. IEEE Int'l Conf. Image Processing, pp. 407-411, 1994.
[49] J.R. Smith and S.F. Chang, “VisualSEEk: A Fully Automated Content-Based Image Query System,” ACM Multimedia '96, Nov. 1996.
[50] J.R. Smith and S.F. Chang, “VisualSEEk: A Fully Automated Content-Based Image Query System,” ACM Multimedia '96, Nov. 1996.
[51] W.D. Stromberg and T.G. Farr, “A Fourier-Based Textural Feature Extraction Procedure,” IEEE Trans. Geoscience and Remote Sensing, vol. 24, no. 5, pp. 722-732, 1986.
[52] M.J. Swain and B.H. Ballard, “Color Indexing,” Int'l J. Computer Vision, vol. 7, no. 1, pp. 11-32, 1991.
[53] Y. Tao and W.I. Grosky, “Delaunay Triangulation for Image Object Indexing: A Novel Method for Shape Representation,” Proc. Seventh SPIE Symp. Storage and Retrieval for Image and Video Databases, pp. 631-942, Jan. 1999.
[54] J. Wang, W. Yang, and R. Acharya, "Color Clustering Techniques for Color Content-Based Image Retrieval," Proc. Fourth IEEE Int'l Conf. Multimedia Computing and Systems (ICMCS 97), IEEE CS Press, Los Alamitos, Calif., June 1997, pp. 442-449.
[55] J. Wang and R. Acharya, “Efficient Access to and Retrieval from a Shape Image Database,” Proc. IEEE Workshop Content Based Access of Image and Video Libraries, (CBAIL '98), June 1998.
[56] J. Wang and R. Acharya, “A Vertex Based Shape Coding Approach for Similar Shape Retrieval,” Proc. ACM Symp. Applied Computing, pp. 520-524, Feb. 1998.
[57] W. Wang, J. Yang, and R.R. Muntz, "Sting: A Statistical Information Grid Approach to Spatial Data Mining," Proc. 23rd Int'l Conf. Very Large Databases, Morgan Kaufmann, 1997, pp. 186-195.
[58] D.A. White and R. Jain, “Algorithms and Strategies for Similarity Retrieval,” Technical Report VCL-96-101, Visual Computing Laboratory, Univ. of California, San Diego, 1996.
[59] T. Zhang, R. Ramakrishnan, and M. Livny, "Birch: An Efficient Data Clustering Method for Very Large Databases," Proc. ACM SIGMOD Int'l Conf. Management of Data, ACM Press, 1996, pp. 103-114.
[60] J.M. Zurada, Introduction to Artificial Neural Systems. West Publishing Company, 1992.

Index Terms:
Image databases, content-based retrieval, heterogeneous features, and semantic clustering.
Citation:
Gholamhosein Sheikholeslami, Wendy Chang, Aidong Zhang, "SemQuery: Semantic Clustering and Querying on Heterogeneous Features for Visual Data," IEEE Transactions on Knowledge and Data Engineering, vol. 14, no. 5, pp. 988-1002, Sept.-Oct. 2002, doi:10.1109/TKDE.2002.1033769
Usage of this product signifies your acceptance of the Terms of Use.