This Article 
 Bibliographic References 
 Add to: 
Three-Dimensional Interfaces for Querying by Example in Content-Based Image Retrieval
October-December 2002 (vol. 8 no. 4)
pp. 305-318

Abstract—Image databases are nowadays widely exploited in a number of different contexts, ranging from history of art, through medicine, to education. Existing querying paradigms are based either on the usage of textual strings, for high-level semantic queries or on 2D visual examples for the expression of perceptual queries. Semantic queries require manual annotation of the database images. Instead, perceptual queries only require that image analysis is performed on the database images in order to extract salient perceptual features that are matched with those of the example. However, usage of 2D examples is generally inadequate as effective authoring of query images, attaining a realistic reproduction of complex scenes, needs manual editing and sketching ability. Investigation of new querying paradigms is therefore an important—yet still marginally investigated—factor for the success of content-based image retrieval. In this paper, a novel querying paradigm is presented which is based on usage of 3D interfaces exploiting navigation and editing of 3D virtual environments. Query images are obtained by taking a snapshot of the framed environment and by using the snapshot as an example to retrieve similar database images. A comparative analysis is carried out between the usage of 3D and 2D interfaces and their related query paradigms. This analysis develops on a user test on retrieval efficiency and effectiveness, as well as on an evaluation of users' satisfaction.

[1] N. Aloia, M. Matera, and F. Paternò, “Using Tasks for Improving the Presentations for Database Query Results,” Proc. IEEE Symp. Visual Languages, pp. 121-124, 1997.
[2] N.S. Chang and K.S. Fu, “Query by Pictorial Example,” IEEE Trans. Software Eng., vol. 6, no. 6, pp. 519-524, June 1980.
[3] J. Huang, S.R. Kumar, M. Mitra, W. Zhu, and R. Zabih, Image Indexing Using Color Correlograms Proc. Computer Vision and Pattern Recognition, pp. 762-768, 1997.
[4] Y. Rubner, C. Tomasi, and L. Guibas, “A Metric for Distributions with Applications to Image Databases,” Proc. ICCV '98, pp. 59-66, 1998.
[5] S.K. Chang, Q.Y. Shi, and C.W. Yan, “Iconic Indexing by 2-D Strings,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 3, pp. 413-427, July 1987.
[6] L. Cooper and R. Shepard, “Turning Something over in the Mind,” Scientific Am., pp. 114-120, Dec. 1984.
[7] J.M. Corridoni, A. del Bimbo, and P. Pala, “Image Retrieval by Color Semantics,” Multimedia Systems, vol. 7, pp. 175-183, 1999.
[8] A.D. Bimbo, M. Campanai, and P. Nesi, “A Three-Dimensional Iconic Environment for Image Database Querying,” IEEE Trans. Software Eng., vol. 19, no. 10, pp. 997-1011, Oct. 1993.
[9] A. del Bimbo and P. Pala, “Visual Image Retrieval by Elastic Matching of User Sketches,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 2, pp. 121-132, Feb. 1997.
[10] A.D. Bimbo, E. Vicario, and D. Zingoni, “Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic,” IEEE Trans. Knowledge and Data Eng., vol. 7, no. 4, pp. 609-621, Aug. 1995.
[11] A. Del Bimbo, M. Mugnaini, P. Pala, and F. Turco, “Visual Querying by Color Perceptive Regions,” Pattern Recognition, vol. 31, no. 9, pp. 1241-1253, 1998.
[12] C. Faloutsos, M. Flickner, W. Niblack, D. Petkovic, W. Equitz, and R. Barber, “Efficient and Effective Querying by Image Content,” Research Report 9453, IBM Research Division Almaden Research Center, Aug. 1993.
[13] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” IEEE Computer, 1995.
[14] J.D. Foley et al., Computer Graphics: Principles and Practice, Second Edition in C, Addison-Wesley, Reading, Mass., 1995.
[15] B. Funt and G. Finlayson, "Color Constant Color Indexing," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 5, pp. 522-529, May 1995.
[16] T. Gevers and A. Smeulders, “A Comparative Study of Several Color Models for Color Image Invariant Retrieval,” Proc. First Int'l Workshop Image Databases and Multimedia Search, pp. 17-27, Aug. 1996.
[17] V.N. Gudivada, "Spatial Knowledge Representation and Retreival in 3D Image Databases," Int'l Conf. Multimedia and Computing Systems, IEEE CS Press, Los Alamitos, Calif., May, 1995, pp. 90-98.
[18] J. Hafner, H.S. Sawhney, W. Equitz, M. Flickner, and W. Niblack, “Efficient Color Histogram Indexing for Quadratic Form Distance Functions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 7, pp. 729-736, July 1995.
[19] J. Hildebrandt and K. Tang, “A Two and Three Dimensional Ship Database Application,” Proc. Workshop Spatial Reasoning, Aug. 1993.
[20] K. Hirata and T. Kato, “Query by Visual Example,” Advances in Database Technology EDBT '92, Third Int'l Conf. Extending Database Technology, 1992.
[21] C.E. Jacobs and A. Finkelstein, S.H. Salesin, “Fast Multiresolution Image Querying,” Proc. SIGGRAPH, 1995.
[22] K.C. Liang and C.C.J. Kuo, “Progressive Image Indexing and Retrieval Based on Embedded Wavelet Coding,” Proc. Int'l Conf. Image Processing (ICIP '97), vol. 1, pp. 572-575, 1997.
[23] G. Marchionini and H. Maurer, “The Roles of Digital Libraries in Teaching and Learning,” Comm. ACM, vol. 38, no. 4, pp. 67-75, Apr. 1995.
[24] P. Haeberli and M. Segal, “Texture Mapping as Fundamental Drawing Primitive,” Proc. Fourth Eurographics Workshop Rendering, M. Cohen, C. Puech, and F. Sillion eds., 1993.
[25] J. Nielsen, Usability Engineering, Academic Press, New York, 1993.
[26] Y. Rui, T.S. Huang, M. Ortega, and S. Mehrotra, “Relevance Feednack: A Power Tool for Interactive Conten-Based Image Retrieval,” IEEE Trans. Circuits, and Video Technology, Sept. 1998.
[27] C.R. Shyu, A.C. Kak, C.E. Brodley, C. Pavlopoulou, M.F. Chyan, and L.S. Broderick, “A Web-Based CBIR-Assisted Learning Tool for Radiology Education—Anytime and Anyplace,” Proc. IEEE Int'l Conf. Multimedia and Expo 2000 (ICME 2000), July 2000.
[28] A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, Content-Based Image Retrieval at the End of the Early Years IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, Dec. 2000.
[29] J.R. Smith and S.F. Chang, “VisualSEEk: A Fully Automated Content-Based Image Query System,” ACM Multimedia '96, Nov. 1996.
[30] Spacemouse,http:/, 2002.
[31] M.J. Swain and B.H. Ballard, “Color Indexing,” Int'l J. Computer Vision, vol. 7, no. 1, pp. 11-32, 1991.
[32] A. Nagasaka and Y. Tanaka, “Automatic Video Indexing and Full Video Search for Object Appearances,” IFIP Trans. Visual Database Systems II, Knuth and Wegner, eds., pp. 113-127, 1992.
[33] M. Stricker and A. Dimai, “Color Indexing with Weak Spatial Constraints” Proc. SPIE Conf. Storage and Retrieval for Image Databases, 1996.
[34] L. Taycher, M. La Cascia, and S. Sclaroff, “Image Digestion and Relevance Feedback in the ImageRover WWW Search Engine,” Proc. Visual '97, Dec. 1997.
[35] A. Vellaikal and C.C.J. Kuo, “Content-Based Retrieval Using Multiresolution Histogram Representation,” Digital Image Storage Archiving Systems, vol. 2602, pp. 312-323, Oct. 1995

Index Terms:
Content-based image retrieval, 3D user interfaces.
Jürgen Assfalg, Alberto Del Bimbo, Pietro Pala, "Three-Dimensional Interfaces for Querying by Example in Content-Based Image Retrieval," IEEE Transactions on Visualization and Computer Graphics, vol. 8, no. 4, pp. 305-318, Oct.-Dec. 2002, doi:10.1109/TVCG.2002.1044517
Usage of this product signifies your acceptance of the Terms of Use.