This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Survey on Content-Based Retrieval for Multimedia Databases
January/February 1999 (vol. 11 no. 1)
pp. 81-93

Abstract—Conventional database systems are designed for managing textual and numerical data, and retrieving such data is often based on simple comparisons of text/numerical values. However, this simple method of retrieval is no longer adequate for the multimedia data, since the digitized representation of images, video, or data itself does not convey the reality of these media items. In addition, composite data consisting of heterogeneous types of data also associates with the semantic content acquired by a user's recognition. Therefore, content-based retrieval for multimedia data is realized taking such intrinsic features of multimedia data into account. Implementation of the content-based retrieval facility is not based on a single fundamental, but is closely related to an underlying data model, a priori knowledge of the area of interest, and the scheme for representing queries. This paper surveys recent studies on content-based retrieval for multimedia databases from the point of view of three fundamental issues. Throughout the discussion, we assume databases that manage only nontextual/numerical data, such as image or video, are also in the category of multimedia databases.

[1] J.F. Allen, “Maintaining Knowledge about Temporal Intervals,” Comm. ACM, vol. 26, no. 11, pp. 832–843, 1983.
[2] M. Atkinson, F. Bancilhon, D. DeWitt, K. Dittrich, D. Maier, and S. Zdonik, "The Object-Oriented Database System Manifesto," Proc. First Int'l Conf. Deductive and Object-Oriented Databases, pp. 40-57, 1989.
[3] J.R. Bach, S. Paul, and R. Jain, “A Visual Information Management System for the Interactive Retrieval of Faces,” IEEE Trans. Knowledge and Data Eng., vol. 5, no. 4, pp. 619-628, 1993.
[4] A.D. Bimbo, M. Campanai, and P. Nesi, "3D Visual Query Language for Image Databases," J. Visual Languages and Computing, vol. 3, no. 3, pp. 257-271, 1992.
[5] A.D. Bimbo, E. Vicario, and D. Zingoni, “Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic,” IEEE Trans. Knowledge and Data Eng., vol. 7, no. 4, pp. 609-621, Aug. 1995.
[6] A.D. Bimbo, P. Pala, and S. Santini, "Image Retrieval by Elastic Matching of Shapes and Image Patterns," Proc. IEEE Int'l Conf. Multimedia Computing and Systems, pp. 215-218, June 1996.
[7] A. Celentano, M.G. Fugini, and S. Pozzi, "Knowledge-Based Rtrieval of Office Documents," Proc. 13th Int'l Conf. Research and Development in Information Retrieval, pp. 241-254, Sept. 1990.
[8] S.K. Chang, Q.Y. Shi, and C.W. Yan, “Iconic Indexing by 2-D Strings,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 3, pp. 413-427, July 1987.
[9] S.-F. Chang, W. Chen, H.E. Meng, H. Sundaram, and D. Zong, "VideoQ: An Automated Content Based Video Search System Using Visual Cues," Proc. ACM Multimedia, pp. 313-324,Seattle, 1994.
[10] J.M. Corridoni, A.D. Bimbo, S. De Magistris, and E. Vicario, "A Visual Language for Color-Based Painting Retrieval," Proc. Int'l Symp. Visual Languages, pp. 68-75, 1996.
[11] G. Costagliola, M. Tucci, and S.K. Chang, "Representing and Retrieving Symbolic Pictures by Spatial Relations," Visual Database Systems, vol. II, E. Knuth and L.M. Wegner, eds., Elsevier, pp. 55-65, 1991.
[12] I.F. Cruz and W.T. Lucas, "A Visual Approach to Multimedia Querying and Presentation," Proc. ACM Multimedia, pp. 109-120, 1997.
[13] Y.F. Day, S. Dagstas, and A. Ghafoor, “Spatio-Temporal Modeling of Video Data for On-Line Object-Oriented Query Processing,” Proc. IEEE Int'l Conf. Multimedia (ICMCS '95), pp. 98-105, 1995.
[14] M.J. Egenhofer, "Query Processing in Spatial-Query-by-Sketch," J. Visual Languages and Computing, vol. 8, no. 4, pp. 403-424, 1997.
[15] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” IEEE Computer, 1995.
[16] A. Ghias, J. Logan, and D. Chamberlin, "Query by Humming," Proc. ACM Multimedia, pp. 231-236, 1995.
[17] S. Gibbs, L. Dami, and D. Tsichritzis, "An Object-Oriented Framework for Multimedia Composition and Synchronization," Proc. Multimedia—First Eurographic Workshop Systems, Interaction, and Applications, W.T. Hewtt et al., eds., pp. 101-111,Stockholm, Springer-Verlag, 1992.
[18] Y. Gong, H. Zhang, H.C. Chuan, and M. Sakauchi, "An Image Database System with Content Capturing and Fast Image Indexing Abilities," Proc. Int'l Conf. Multimedia Computing and Systems, pp. 121-130, May 1994.
[19] V.N. Gudivada and G.S. Jung, "An Algorithm for Content-Based retrieval in Multimedia Databases," Proc. Int'l Conf. Multimedia Computing and Systems, pp. 193-200, 1996.
[20] A. Gupta, T. Weymouth, and R. Jain, "Semantic Queries in Image Databases," Visual Database Systems, vol. II, E. Knuth and L.M. Wegner, eds., Elsevier, pp. 204-218, 1991.
[21] A. Gupta, T. Weymouth, and R. Jain, "Semantic Queries With Pictures: The VIMSYS Model," Proc. 17th Int'l Conf. Very Large Data Bases, pp. 69-79, Sept. 1991.
[22] V. Haarslev and M. Wessel, "Querying GIS With Animated Spatial Sketches," Proc. Int'l Symp. Visual Languages, pp. 201-208, Sept. 1997.
[23] S.A. Hawamdeh, B.C. Ooi, R. Price, T.H. Tng, Y.H. Ang, and L. Hui "Nearest Neighbour Searching in a Picture Archive System," Proc. Int'l Conf. Multimedia Information Systems, McGraw-Hill, pp. 17-33, 1991.
[24] P. Hopner, "Synchronizing the Presentation of Multimedia Objects—ODA Extensions," Multimedia Systems, Interaction, and Application, pp. 87-100, Springer-Verlag, 1992.
[25] C.C. Hsu, W.W. Chu, and R.K. Taira, “A Knowledge-Based Approach for Retrieving Images by Content,” IEEE Trans. Knowledge and Data Eng., vol. 8, no. 4, pp. 522-532, 1996.
[26] M. Iino, Y.F. Day, and A. Ghafoor, "An Object-Oriented Model for Spatiotemporal Synchronization of Multimedia Information," Proc. of the IEEE Multimedia Conf., IEEE CS Press, Los Alamitos, Calif., 1994, pp. 110-119.
[27] T. Kato, T. Kurita, H. Shimogaki, T. Mizutori, and K. Fujimura, "A Cognitive Approach to Visual Interaction," Proc. Int'l Conf. Multimedia Information Systems, pp. 109-120, McGraw-Hill, 1991.
[28] A. Klinger and A. Pizano, "Visual Structure and Databases," Visual Database Systems, T.L. Kunii, ed., pp. 3-25, Elsevier, 1989.
[29] E.B.W. Lieutenant and J.R. Driscoll, "Incorporating A Semantic Analysis into A Document Retrieval Strategy," Proc. ACM/SIGIR Conf. Research and Development Information Retrieval, pp. 270-279, Oct. 1991.
[30] T.D.C. Little and A. Ghafoor, “Interval-Based Conceptual Models for Time-Dependent Multimedia Data,” IEEE Trans. Knowledge and Data Eng., vol. 5, no. 4, pp. 551-563, Aug. 1993.
[31] T.D.C. Little, G. Ahanger, R.J. Folz, J.F. Gibbon, F.W. Reeve, D.H. Schelleng, and D. Venkatesh, “A Digital On-Demand Video Service Supporting Content-based Queries,” Proc. ACM Multimedia Conf., pp. 427-436, Aug. 1993.
[32] Z.Q. Liu and J.P. Sun, "Structured Image Retrieval," J. Visual Languages and Computing, vol. 8, no. 3, pp. 333-357, 1997.
[33] K. Melih and R. Gonzalez, "Audio Retrieval Using Perceptually Based Structures," Proc. Int'l Conf. Multimedia Computing and Systems, pp. 338-347, 1998.
[34] B. Meyer, "Pictorial Deduction in Spatial Information Systems," Proc. Int'l Symp. Visual Languages, pp. 23-30, 1994.
[35] A. Ono, M. Amano, M. Hakaridani, T. Satou, and M. Sakauchi, "A Flexible Content-Based Image Retrieval System with Combined Scene Sescription Keyword," Proc. Int'l Conf. Multimedia Computing and Systems, pp. 201-208, 1996.
[36] E. Oomoto, “Design and Implementation of a Video-Object Database System,” IEEE Trans. Knowledge and Data Eng., vol. 5, no. 4, pp. 629-643, Aug. 1993.
[37] K. Otsuji and Y. Tonomura, "Projection Detecting Filter for Video Cut Detection," Proc. ACM Multimedia 93, ACM Press, New York, 1993, pp. 251-257.
[38] A.P. Sistla, C.T. Yu, and R. Haddad, “Reasoning About Spatial Relationships in Picture Retrieval Systems, Proc. 1994 Int’l Conf. Very Large Databases, Morgan Kaufmann, San Mateo, Calif., 1994.
[39] A.P. Sistla, C. Yu, C. Liu, and K. Liu, "Similarity Based Retrieval of Pictures Using Indices on Spatial Relationships," Proc. Int'l Conf. Very Large Databases, pp. 619-629, Sept. 1995.
[40] P.J. Smith, S.J. Shute, and D. Galdes, "In Search of Knowledge-Based Search Tactics," Proc. 12th Int'l Conf. Research and Development in Information Retrieval, pp. 3-10, 1989.
[41] Y. Theodoridis, M. Vazirgiannis, and T. Sellis, "Spatio-Temporal Indexing for Large Multimedia Applications," Proc. Int'l Conf. Multimedia Computing and Systems, pp. 441-448, 1996.
[42] D. Toman, "Point vs. Interval-Based Query Languages for Temporal Databases," Proc. Fifth ACM SIGACT/MOD/ART Symp. Principles of Database Systems, pp. 58-67, 1996.
[43] K. Tsuda, K. Yamamoto, M. Hirakawa, and T. Ichikawa, "MORE: An Object-Oriented Data Model with A Facility for Changing Object Structures," IEEE Trans. Knowledge and Data Eng., vol. 3, no. 4, pp. 444-460, 1991.
[44] R. Weiss, A. Duda, and D.K. Gifford, “Content-Based Access to Algebraic Video,” Proc. Int'l Conf. Multimedia Computing and Systems, pp. 140-151, May 1994.
[45] E. Wold et al., "Content-Based Classification, Search, and Retrieval of Audio," IEEE MultiMedia, Vol. 3, No. 3, 1966, pp. 27-36.
[46] X. Wu and T. Ichikawa, "KDA: A Knowledge-Based Database Assistant with A Query Guiding Facility," IEEE Trans. Knowledge and Data Eng., vol. 4, no. 5, pp. 443-453, 1994.
[47] W. Hsu, T.S. Chua, and H.K. Pung, "An Integrated Color-Spatial Approach to Content-Based Image Retrieval," Proc. ACM Multimedia Conf., pp. 305-313, 1995.
[48] M.M. Yeung, B-L Yeo, and B. Liu, "Extracting Story Units from Long Programs for Video Browsing and Navigation," Proc. Third IEEE Int'l Conf. Multimedia Computing and Systems, June 1996.
[49] A. Yoshitaka, S. Kishida, M. Hirakawa, and T. Ichikawa, "Knowledge-Assisted Content Based Retrieval for Multimedia Databases," IEEE Multimedia, pp. 12-21, Winter 1994.
[50] A. Yoshitaka, Y. Hosoda, M. Yoshimitsu, M. Hirakawa, and T. Ichikawa, "VIOLONE:Video Retrieval By Motion Example," J. Visual Languages and Computing, vol. 7, no. 4, pp. 423-443, 1996.
[51] A. Yoshitaka, T. Ishii, M. Hirakawa, and T. Ichikawa, "Content-Based Retrieval of Video Data by the Grammar of the Film," Proc. Int'l Symp. Visual Languages, pp. 314-321, Sept. 1997.
[52] G.P. Zarri, "Conceptual Representation for Knowledge Bases and 'Intelligent' Information Retrieval Systems," Proc. 11th Int'l Conf. Research and Development in Information Retrieval, pp. 551-565, 1988.
[53] M.M. Zloof, "QBE/OBE: A Language for Office and Business Automation," Computer, vol. 14, no. 5, pp. 13-22, 1981.

Index Terms:
Multimedia databases, content-based retrieval, spatio-temporal relation, query-by-example, knowledge.
Citation:
Atsuo Yoshitaka, Tadao Ichikawa, "A Survey on Content-Based Retrieval for Multimedia Databases," IEEE Transactions on Knowledge and Data Engineering, vol. 11, no. 1, pp. 81-93, Jan.-Feb. 1999, doi:10.1109/69.755617
Usage of this product signifies your acceptance of the Terms of Use.