This Article 
 Bibliographic References 
 Add to: 
Content-Based Indexing of Multimedia Databases
November-December 1997 (vol. 9 no. 6)
pp. 978-989

Abstract—Content-based retrieval of multimedia database calls for content-based indexing techniques. Different from conventional databases, where data items are represented by a set of attributes of elementary data types, multimedia objects in multimedia databases are represented by a collection of features; similarity of object contents depends on context and frame of reference; and features of objects are characterized by multimodal feature measures. These lead to great challenges for content-based indexing. On the other hand, there are special requirements on content-based indexing: To support visual browsing, similarity retrieval, and fuzzy retrieval, nodes of the index should represent certain meaningful categories. That is to say that certain semantics must be added when performing indexing. ContIndex, the context-based indexing technique presented in this paper, is proposed to meet these challenges and special requirements. The indexing tree is formally defined by adapting a classification-tree concept. Horizontal links among nodes in the same level enhance the flexibility of the index. A special neural-network model, called Learning based on Experiences and Perspectives (LEP), has been developed to create node categories by fusing multimodal feature measures. It brings into the index the capability of self-organizing nodes with respect to certain context and frames of reference. An icon image is generated for each intermediate node to facilitate visual browsing. Algorithms have been developed to support multimedia object archival and retrieval using ContIndex. ContIndex has been successfully applied to two applications: A facial image retrieval system, CAFIIR, and a trademark archival and registration system, STAR.

[1] J.R. Bach, S. Paul, and R. Jain, “A Visual Information Management System for the Interactive Retrieval of Faces,” IEEE Trans. Knowledge and Data Eng., vol. 5, no. 4, pp. 619-628, 1993.
[2] W.I. Grosky, “Multi-Media Information Systems.” IEEE Multimedia, vol. 1, no. 1, Mar. 1994.
[3] A. Pentland, R.W. Picard, and S. Sclaroff, "Photobook: Tools for Content Based Manipulation of Image Databases," Proc. SPIE Conf. Storage and Retrieval of Image and Video Databases, vol. 2, no. 2,185, Feb. 1994.
[4] E.G.M. Petrakis and S.C. Orphanoudakis, "Methodology for the Representation, Indexing and Retrieval of Images by Content," Image and Vision Computing, vol. 11, pp. 504-520, 1993.
[5] J.K. Wu, A.D. Narasimhalu, B.M. Mehtre, C.P. Lam, and Y.J. Gao, “CORE: A Content Based Retrieval System for Multimedia Information Systems,” Multimedia Systems, vol. 3, pp. 25-41, 1995.
[6] J.K. Wu, Y.H. Ang, C.P. Lam, H.H. Loh, and A.D. Narasimhalu, "Inference and Retrieval of Facial Images," ACM Multimedia J., vol. 2, no. 1, pp. 1-14, 1994.
[7] J.K. Wu, C.P. Lam, B.M. Mehtre, Y.J. Gao, and A. D. Narasimhalu, "Content-Based Retrieval For Trademark Registration," Multimedia Tools and Applications, vol. 3, no. 3, pp. 245-267, 1996.
[8] C. Faloutsos, M. Flickner, W. Niblack, D. Petkovic, W. Equitz, and R. Barber, "Efficient and Effective Querying by Image Content," Technical Report, IBM Research Division, Almaden Research Center, RJ 9453 (83074), Aug. 1993.
[9] S.K. Chang, C.W. Yan, D.C. Dimitroff, and T. Arndt, “An Intelligent Image Database System,” IEEE Trans. Software Eng., vol. 14, no. 5, pp. 681-688, May 1988.
[10] W.I. Gorsky and R. Mehrotra, "Index-Based Object Recognition in Pictorial Data Management," Computer Vision, Graphics, and Image Processing, vol. 52, pp. 416-436, 1990.
[11] P. Lewis et al., "Media-Based Navigation with Generic Links," Proc. ACM Hypertext 96, ACM Press, New York, 1996, pp. 215-223.
[12] K. Hirata Media-Based Navigation for Hypermedia Systems, ACM Hypertext '93, pp. 159-173, 1993.
[13] T. Kohonen, "The Self-Organizing Map," Proc. IEEE, vol. 78, pp. 1,464-1,480, 1990.
[14] J.L. McClelland and D.E. Rumelhart, Explorations in Parallel Distributed Processing.Cambridge, Mass.: MIT Press, 1986.
[15] A. Tversky, "Features of Similarity," Psychological Rev., vol. 84, pp. 327-352, 1977.
[16] J.-K. Wu, Neural Networks and Simulation.New York: Marcel Dekker, 1994.
[17] Y.H. Ang, "Enhanced Primal Sketch Face Recognition," Looking at People: Recognition and Interpretation of Human Action, A. Pentland, ed., Proc 13th Int'l Joint Conf. Artificial Intelligence, Chambery, Savoie, France, 1994.
[18] K. Hornik, M. Stinchcombe, and H. White, “Multilayer Feedforward Networks are Universal Approximations,” Neural Networks, vol. 2, pp. 359-366, 1989.
[19] C.Y. Suen, “Computer Recognition of Unconstrained Handwritten Numerals,” Proc. IEEE, vol. 80, pp. 1,162-1,180, 1992.
[20] J.-K. Wu, F. Gao, and P. Yang, "Model-Based 3D Object Recognition," Proc. Second Int'l Conf. Automation, Robotics, and Computer Vision,Singapore, Sept. 1992.
[21] A.L. Yuile, "Deformable Templates for Face Recognition," J. Cognitive Neuroscience, vol. 3, pp. 59-70, 1991.
[22] P.A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach.Englewood Cliffs, N.J.: Prentice Hall, 1982.
[23] J.K. Wu, C.P. Lam, and G. Senthilkumar, "Evaluation of Feature Measures and Similarity Measures for Content-Based Retrieval," Proc. SPIE Symp. Digital Image Storage and Archiving Systems, vol. 2,606, Philadelphia, Oct. 1995.

Index Terms:
Indexing, content-based retrieval, multimedia, image database, image analysis, neural networks, fusion of multiple feature measures.
Jian-Kang Wu, "Content-Based Indexing of Multimedia Databases," IEEE Transactions on Knowledge and Data Engineering, vol. 9, no. 6, pp. 978-989, Nov.-Dec. 1997, doi:10.1109/69.649320
Usage of this product signifies your acceptance of the Terms of Use.