This Article 
 Bibliographic References 
 Add to: 
SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries
September 2001 (vol. 23 no. 9)
pp. 947-963

—The need for efficient content-based image retrieval has increased tremendously in many application areas such as biomedicine, military, commerce, education, and Web image classification and searching. We present here SIMPLIcity (Semantics-sensitive Integrated Matching for Picture LIbraries), an image retrieval system, which uses semantics classification methods, a wavelet-based approach for feature extraction, and integrated region matching based upon image segmentation. As in other region-based retrieval systems, an image is represented by a set of regions, roughly corresponding to objects, which are characterized by color, texture, shape, and location. The system classifies images into semantic categories, such as textured-nontextured, graph-photograph. Potentially, the categorization enhances retrieval by permitting semantically-adaptive searching methods and narrowing down the searching range in a database. A measure for the overall similarity between images is developed using a region-matching scheme that integrates properties of all the regions in the images. Compared with retrieval based on individual regions, the overall similarity approach 1) reduces the adverse effect of inaccurate segmentation, 2) helps to clarify the semantics of a particular region, and 3) enables a simple querying interface for region-based image retrieval systems. The application of SIMPLIcity to several databases, including a database of about 200,000 general-purpose images, has demonstrated that our system performs significantly better and faster than existing ones. The system is fairly robust to image alterations.

[1] M.C. Burl, M. Weber, and P. Perona, “A Probabilistic Approach to Object Recognition Using Local Photometry and Global Geometry,” Proc. European Conf. Computer Vision, pp. 628-641, June 1998.
[2] C. Carson, M. Thomas, S. Belongie, J.M. Hellerstein, and J. Malik, “Blobworld: A System for Region-Based Image Indexing and Retrieval,” Proc. Visual Information Systems, pp. 509-516, June 1999.
[3] I. Daubechies,“Ten lectures on wavelets,” SIAM CBMS-61, 1992.
[4] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” IEEE Computer, 1995.
[5] M. Fleck, D.A. Forsyth, and C. Bregler, “Finding Naked People,” Proc. European Conf. Computer Vision, vol. 2, pp. 593-602, 1996.
[6] A. Gersho, “Asymptotically Optimum Block Quantization,” IEEE Trans. Information Theory, vol. 25, no. 4, pp. 373-380, July 1979.
[7] A. Gupta and R. Jain, “Visual Information Retrieval,” Comm. ACM, vol. 40, no. 5, pp. 70-79, May 1997.
[8] J.A. Hartigan and M.A. Wong, “Algorithm AS136: A k-means Clustering Algorithm,” Applied Statistics, vol. 28, pp. 100-108, 1979.
[9] R. Jain, S.N.J. Murthy, P.L.-J. Chen, and S. Chatterjee, “Similarity Measures for Image Databases,” Proc. SPIE, vol. 2420, pp. 58-65, Feb. 1995.
[10] K. Karu, A.K. Jain, and R.M. Bolle, “Is There any Texture in the Image?” Pattern Recognition, vol. 29, pp. 1437-1446, 1996.
[11] W.Y. Ma and B.S. Manjunath, “NETRA: A Toolbox for Navigating Large Image Databases,” Proc. IEEE Int'l Conf. Image Processing, 1997.
[12] T.P. Minka and R.W. Picard, “Interactive Learning Using a Society of Models,” Pattern Recognition, vol. 30, no. 3, p. 565, 1997.
[13] S. Mukherjea, K. Hirata, and Y. Hara, “AMORE: A World Wide Web Image Retrieval Wngine,” Proc. World Wide Web, vol. 2, no. 3, pp. 115-132, 1999.
[14] A. Natsev, R. Rastogi, and K. Shim, “Walrus: A Similarity Retrieval Algorithm for Image Databases,” Proc. ACM SIGMOD Conf. Management of Data, pp. 395-406, 1999.
[15] A. Pentland, R.W. Picard, and S. Sclaroff, “Photobook: Tools for Content-Based Manipulation of Image Databases,” Proc. SPIE, vol. 2185, pp. 34-47, Feb. 1994.
[16] E.G.M. Petrakis and C. Faloutsos, “Similarity Searching in Medical Image Databases,” IEEE Trans. Knowledge and Data Eng., vol. 9, no. 3, pp. 435-447, May/June 1997.
[17] R.W. Picard and T. Kabir, "Finding Similar Patterns in Large Image Databases," Proc. IEEE Conf. Acoustics, Speech, and Signal Processing, pp. V-161-V-164,Minneapolis, 1993.
[18] Y. Rubner, L.J. Guibas, and C. Tomasi, “The Earth Mover's Distance, Multi-Dimensional Scaling, and Color-Based Image Retrieval,” Proc. DARPA Image Understanding Workshop, pp. 661-668, May 1997.
[19] G. Sheikholeslami, W. Chang, and A. Zhang, “Semantic Clustering and Querying on Heterogeneous Features for Visual Data,” Proc. Sixth ACM Int'l Multimedia Conf. (ACM Multimedia '98), pp. 3-12, Sept. 1998.
[20] J. Shi and J. Malik, “Normalized Cuts and Image Segmenation,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 731-737, June 1997.
[21] J.R. Smith and S.F. Chang, “VisualSEEk: A Fully Automated Content-Based Image Query System,” ACM Multimedia '96, Nov. 1996.
[22] J.R. Smith and C.S. Li, “Image Classification and Querying Using Composite Region Templates,” J. Computer Vision and Image Understanding, vol. 75, nos. 1-2, pp. 165-174, 1999.
[23] S. Stevens, M. Christel, and H. Wactlar, “Informedia: Improving Access to Digital Video,” Interactions, vol. 1, no. 4, pp. 67-71, 1994.
[24] M. Szummer and R.W. Picard, “Indoor-Outdoor Image Classification,” Proc. IEEE Int'l Workshop Content-Based Access of Image and Video Databases, 1998.
[25] M. Unser, Texture Classification and Segmentation Using Wavelet Frames IEEE Trans. Image Processing, vol. 4, no. 11, pp. 1549-1560, Nov. 1995.
[26] A. Vailaya, A. Jain, and H. Zhang, "On Image Classification: City vs. Landscape," Proc. IEEE Workshop on Content-Based Access of Image and Video Libraries, IEEE Press, Piscataway, N.J., 1998, pp. 3-8.
[27] J.Z. Wang, J. Li, R. M. Gray, and G. Wiederhold, “Unsupervised Multiresolution Segmentation for Images with Low Depth of Field,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 1, pp. 85-91, Jan. 2001.
[28] J.Z. Wang, G. Wiederhold, O. Firschein, and X.W. Sha, “Content-Based Image Indexing and Searching Using Daubechies' Wavelets,” Int'l J. Digital Libraries, vol. 1, no. 4, pp. 311-328, 1998.
[29] J.Z. Wang, J. Li, G. Wiederhold, and O. Firschein, “System for Screening Objectionable Images,” Computer Comm., vol. 21, no. 15, pp. 1355-1360, 1998.
[30] J.Z. Wang and M.A. Fischler, “Visual Similarity, Judgmental Certainty and Stereo Correspondence,” Proc. DARPA Image Uunderstanding Workshop, 1998.

J.Z. Wang, J. Li, G. Wiederhold, "SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 9, pp. 947-963, Sept. 2001, doi:10.1109/34.955109
Usage of this product signifies your acceptance of the Terms of Use.