This Article 
 Bibliographic References 
 Add to: 
Techniques and Systems for Image and Video Retrieval
January/February 1999 (vol. 11 no. 1)
pp. 56-63

Abstract—Storage and retrieval of multimedia has become a requirement for many contemporary information systems. These systems need to provide browsing, querying, navigation, and, sometimes, composition capabilities involving various forms of media. In this survey, we review techniques and systems for image and video retrieval. We first look at visual features for image retrieval such as color, texture, shape, and spatial relationships. The indexing techniques are discussed for these features. Nonvisual features include captions, annotations, relational attributes, and structural descriptions. Temporal aspects of video retrieval and video segmentation are discussed next. We review several systems for image and video retrieval including research, commercial, and World Wide Web-based systems. We conclude with an overview of current challenges and future trends for image and video retrieval.

[1] P. Alshuth, T. Hermes, J. Kreyb, and M. Roper, "Video Retrieval With IRIS," Proc. ACM Multimedia Conf., p. 421,Boston, 1996.
[2] A. Aslandogan, C. Thier, C. Yu, C. Liu, and K. Nair, “Design, Implementation and Evaluation of SCORE (a System for COntent based REtrieval of Pictures),” IEEE Data Eng., 1995.
[3] Y.A. Aslandogan, C. Thier, T.C. Yu, J. Zou, and N. Rishe, "Using Semantic Contents and WordNet in Image Retrieval," Proc. ACM SIGIR Conf., 1997.
[4] A. La Corte, A. Lombardo, S. Palazzo, and G. Schembra, "Control of Perceived Quality of Service in Multimedia Retrieval Services: Prediction-Based Mechanisms vs. Compensation Buffers," Multimedia Systems, vol. 6, pp. 102-112, ACM/Springer, 1998.
[5] S. Berchtold, C. Bohm, and H.-P. Kriegel, "The Pyramid-Technique: Towards Breaking the Curse of Dimensionality," Proc. ACM SIGMOD, pp. 142-153, 1998.
[6] M. L. Cascia and E. Ardizzone, JACOB: Just a Content-Based Query System for Video Databases Proc. Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 1216-1219, 1996.
[7] J.D. Courtney, "Automatic, Object-Based Indexing for Assisted Analysis of Video Data," Proc. ACM Multimedia Conf.,Boston, pp. 423-424, 1996.
[8] S.-F. Chang, J.R. Smith, and J. Meng, "Efficient Techniques for Feature-Based Image/Video Access and Manipulation," Proc. 33rd Ann. Clinic on Library Applications of Data Processing Image Access and Retrieval, invited paper, Mar. 1996.
[9] S.F. Chang, J. Smith, and H. Wang, "Automatic Feature Extraction and Indexing for Content-Based Visual Query," Columbia Univ., Technical Report No. CU/CTR 414-95-20, Jan. 1995.
[10] S.K. Chang, Q. Shi, and C. Yan, "Iconic Indexing by 2-D String," IEEE Trans. Pattern Analysis and Machine Intelligence, 1987.
[11] R. Chellapa, C.L. Wilson, and S. Sirohey, "Human and Machine Recognition of Faces: A Survey," Proc. IEEE, vol. 83, no. 5, May 1995.
[12] M.J. Egenhofer, "What's Special About Spatial? Database Requirements for Vehicle Navigation in Geographic Space," Proc. ACM SIGMOD, pp. 398-402, 1993.
[13] C. Faloutsos, R. Barber, M. Flicker, J. Hafner, W. Niblack, and W. Equitz, "Efficient and effective querying by image content," J. Intell. Information Systems," vol. 3, pp. 231-262, 1994.
[14] D. Le Gall, “MPEG: A Video Compression Standard for Multimedia Applications,” Comm. ACM, Apr. 1991.
[15] A. Gupta, Visual Information Retrieval Technology, A VIRAGE Perspective, white paper, Virage, 1995.
[16] A. Guttman, "R-Trees: A Dynamic Index Structure for Spatial Searching," Proc. ACM SIGMOD Conf., pp. 47-57, June 1984.
[17] D. Hang, B. Cheng, and R. Acharya, "Texture-Based Image Retrieval Using Fractal Codes," Technical Report No. 95-19, Dept. of Computer Science, State Univ. of New York at Buffalo, Aug. 1995.
[18] G. Halin, M. Crehange, and P. Kerekes, "Machine Learning and Vectorial Matching for an Image Retrieval Model: EXPRIM and the System RIVAGE," Proc. ACM-SIGIR, pp. 99-114,Brussles, 1990.
[19] W. Hsu, T.S. Chua, and H.K. Pung, "An Integrated Color-Spatial Approach to Content-Based Image Retrieval," Proc. ACM Multimedia Conf., pp. 305-313, 1995.
[20] C.C. Hsu, W.W. Chu, and R.K. Taira, “A Knowledge-Based Approach for Retrieving Images by Content,” IEEE Trans. Knowledge and Data Eng., vol. 8, no. 4, pp. 522-532, 1996.
[21] G.S. Jung and V. Gudivada, "Adaptive Query reformulation in Attribute Based Image Retrieval," Intelligent Systems, pp. 763-774, 1995.
[22] R. Jain, W.N. Martin, and J.K. Aggarwal, "Segmentation Through the Detection of Changes Due to Motion," Computer Graphics and Image Processing, vol. 11, pp. 13-34, 1979.
[23] A.K. Jain, Y. Zhong, and S. Lakshmanan, Object Matching Using Deformable Templates IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 3, pp. 267-278, Mar. 1996.
[24] V. Kobla, D. Doermann, and K.-I. Lin, "Archiving, Indexing, and Retrieval of Video in the Compressed Domain," Multimedia Storage and Archival Systems, pp. 78-89, SPIE, Nov. 1996.
[25] T. Kato, T. Kurita, N. Otsu, and K. Hirata, “A Sketch Retrieval Method for Full Color Image Database—Query by Visual Example,” Proc. ICPR, Computer Vision and Applications, pp. 530-533, 1992.
[26] W.-S. Li, K.S. Candan, K. Hirata, and Y. Hara, "SEMCOG: An Object-Based Image Retrieval System and its Visual Query Interface," Proc. ACM SIGMOD, pp. 521-524, June 1997.
[27] R. Lienhart, "Automatic Text Recognition for Video Indexing," Proc. ACM Multimedia Conf., pp. 11-20,Boston, 1996.
[28] M. Krunz and S.K. Tripathi, "Impact of Video Scheduling on Bandwidth Allocation for Multiplexed MPEG Streams," Multimedia Systems, vol. 5, pp. 347-357, ACM/Springer, 1997.
[29] R. Mohan, "Text-Based Search of TV News Stories," Multimedia Storage and Archival Systems, pp. 2-13, SPIE, Nov. 1996.
[30] A. Nagasaka and Y. Tanaka, "Automatic Video Indexing and Full Video Search for Object Appearances," Proc. Conf. Visual Database Systems, pp. 119-133, 1991.
[31] V.E. Ogle, “CHABOT—Retrieval from a Relational Database of Images,” Computer, vol. 28, no. 9, pp. 40-48, Sept. 1995.
[32] E.G.M. Petrakis and C. Faloutsos, "Similarity Searching in Large Image Databases," Dept. of Computer Science, Univ. of Maryland, Technical Report 3388, 1995.
[33] A. Pentland, R. Picard, and S. Sclaroff, "Photobook: Tools for Content-Based Manipulation of Image Databases," Storage and Retrieval of Image and Video Databases II, Paper No. 2185-05, San Jose, Calif., pp. 34-47, SPIE, Feb. 1994.
[34] D. Papadias, T. Sellis, Y. Theodorakis, and M.J. Egenhofer, "Topological Relations in the World of Minimum Bounding Rectangles: A Study with R-Trees," Proc. ACM SIGMOD, pp. 92-103, 1995.
[35] G. Pass, R. Zabih, and J. Miller, “Comparing Images Using Color Coherence Vectors,” Proc. ACM Multimedia '96, pp. 65-73, 1996.
[36] P.V. Rangan and H.M. Vin, "Efficient Storage Techniques for Digital Continuous Multimedia," IEEE Trans. Knowledge and Data Eng. , vol. 5, no. 4, Aug. 1993, pp. 564-573.
[37] H. Rowley, S. Baluja, and K. Kanade, "Human Face Detection in Visual Scenes," Computer Science Dept., Carnegie Mellon Univ., Technical Paper No. CMU-CS-95 158, 1995.
[38] R.K. Srihari, "Automatic Indexing and Content-Based Retrieval of Captioned Images," Computer, vol. 28, no. 9, Sept. 1995, pp. 49-56.
[39] H. Samet, The Design and Analysis of Spatial Data Structures, Addison-Wesley, 1989.
[40] J.R. Smith and S.-F. Chang, "Searching for Images and Videos on the World-Wide Web," CTR Technical Report No. 459-96-25, Columbia Univ., Aug. 1996.
[41] J.R. Smith and S.F. Chang, “VisualSEEk: A Fully Automated Content-Based Image Query System,” ACM Multimedia '96, Nov. 1996.
[42] M.J. Swain, C. Frankel, and V. Athitsos, "WebSeer: An Image Search Engine for the World Wide Web," Dept. of Computer Science, Univ. of Chicago, Technical Report No. TR-96-14, July 1996.
[43] M. Sonka, V. Hlavac, and R. Boyle, Image Processing, Analysis, and Machine Vision, Chapman and Hall, 1993.
[44] A.F. Smeaton and I. Qigley, "Experiments on Using Semantic Distances Between Words in Image Caption Retrieval," Proc. ACM SIGIR Conf., 1996.
[45] A.P. Sistla, C. Yu, and R. Venkatasubrahmanian, "Similarity Based Retrieval of Videos," Proc. IEEE Data Eng. Conf., 1997.
[46] F. Tomita and T. Saburo, Computer Analysis of Visual Textures, Kluwer, 1990.
[47] E.M. Voorhees, "Using WordNet to Disambiguate Word Senses for Text Retrieval," Proc. ACM SIGIR Conf., pp. 171-180, 1993.
[48] R. Weiss, A. Duda, and D.K. Gifford, "Composition and Search with a Video Algebra," IEEE MultiMedia, vol. 2, no. 1, Spring 1995, pp. 12-25.
[49] H. Wactlar, T. Kanade, M.A. Smith, and S.M. Stevens, “Intelligent Access to Digital Video: The Informedia Project,” Computer, vol. 29, no. 5, pp. 46-52, 1996.
[50] C.T. Yu and W. Meng, "Principles of Database Query Processing for Advanced Applications, Data Management Systems, Morgan Kaufmann, 1998.
[51] H. Zhang, A. Kankanhalli, and S. Smoliar, "Automatic Partitioning of Full-Motion Video," Multimedia Systems, Vol. 1, No. 1, 1993, pp. 10-28.
[52] H.J. Zhang, S.Y. Tan, S.W. Smoliar, and Y. Gong, "Automatic Parsing and Indexing of News Video," Multimedia Systems, vol. 2, pp. 256-266, 1995.

Index Terms:
Image and video retrieval, content-based retrieval, multimedia databases, information retrieval.
Y. Alp Aslandogan, Clement T. Yu, "Techniques and Systems for Image and Video Retrieval," IEEE Transactions on Knowledge and Data Engineering, vol. 11, no. 1, pp. 56-63, Jan.-Feb. 1999, doi:10.1109/69.755615
Usage of this product signifies your acceptance of the Terms of Use.