This Article 
 Bibliographic References 
 Add to: 
A Spatio-Temporal Semantic Model for Multimedia Database Systems and Multimedia Information Systems
July/August 2001 (vol. 13 no. 4)
pp. 607-622

Abstract—As more information sources become available in multimedia systems, the development of abstract semantic models for video, audio, text, and image data becomes very important. An abstract semantic model has two requirements: It should be rich enough to provide a friendly interface of multimedia presentation synchronization schedules to the users and it should be a good programming data structure for implementation in order to control multimedia playback. An abstract semantic model based on an augmented transition network (ATN) is presented. The inputs for ATNs are modeled by multimedia input strings. Multimedia input strings provide an efficient means for iconic indexing of the temporal/spatial relations of media streams and semantic objects. An ATN and its subnetworks are used to represent the appearing sequence of media streams and semantic objects. The arc label is a substring of a multimedia input string. In this design, a presentation is driven by a multimedia input string. Each subnetwork has its own multimedia input string. Database queries relative to text, image, and video can be answered via substring matching at subnetworks. Multimedia browsing allows users the flexibility to select any part of the presentation they prefer to see. This means that the ATN and its subnetworks can be included in multimedia database systems which are controlled by a database management system (DBMS). User interactions and loops are also provided in an ATN. Therefore, ATNs provide three major capabilities: multimedia presentations, temporal/spatial multimedia database searching, and multimedia browsing.

[1] J. Allen, Natural Language Understanding, Benjamin/Cummings Publishing, San Francisco, 1995.
[2] Y.Y. Al-Salqan and C.K. Chang, Temporal Relations and Synchronization Agents IEEE Multimedia, vol. 3, pp. 30-39, 1996.
[3] R. Bayer and E. McCreight, “Organization and Maintenance of Large Ordered Indices,” Proc. 1970 ACM-SIGFIDENT Workshop Data Description and Access, pp. 107-141, Nov. 1970.
[4] A.D. Bimbo, E. Vicario, and D. Zingoni, “Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic,” IEEE Trans. Knowledge and Data Eng., vol. 7, no. 4, pp. 609-621, Aug. 1995.
[5] G. Blakowski, J. Huebel, and U. Langrehr, “Tools for Specifying and Executing Synchronized Multimedia Presentations,” Proc. Second Int'l Workshop Network and Operating System Support for Digital Audio and Video, pp. 271-279, 1991.
[6] M. Buchanan and P. Zellweger, “Automatically Generating Consistent Schedules for Multimedia Documents,” ACM Multimedia Systems J., vol. 1, no. 2, pp. 55-67, 1993.
[7] S.K. Chang, C.W. Yan, D.C. Dimitroff, and T. Arndt, “An Intelligent Image Database System,” IEEE Trans. Software Eng., vol. 14, no. 5, pp. 681-688, May 1988.
[8] H. Chang et al., "The Management and Applications of Tele-Action Objects," ACM J. of Multimedia Systems, Vol. 3, No. 5-6, 1995, pp. 204-216.
[9] S.-C. Chen and R.L. Kashyap, “Temporal and Spatial Semantic Models for Multimedia Presentations,” Proc. 1997 Int'l Symp. Multimedia Information Processing, pp. 441-446, 1997.
[10] S.-C. Chen and R.L. Kashyap, “Empirical Studies of Multimedia Semantic Models for Multimedia Presentations,” Proc. 13th Int'l Conf. Computer and Their Applications, pp. 226-229, 1998.
[11] D. Comer, “The Ubiquitous B-Tree,” ACM Computing Surveys, vol. 11, no. 2, pp. 121-137, June 1979.
[12] Y.F. Day, S. Dagtas, M. Iino, A. Khokhar, and A. Ghafoor, “Object-Oriented Conceptual Modeling of Video Data,” Proc. Data Eng. (DE '95), pp. 401-408, 1995.
[13] M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, “Query by Image and Video Content: The QBIC System,” IEEE Computer, 1995.
[14] A. Guttman, “R-Trees: A Dynamic Index Structure for Spatial Searching,” Proc. ACM SIGMOD Conf. Management of Data, 1984.
[15] N. Hirzalla, B. Falchuk, and A. Karmouch, "A Temporal Model for Interactive Multimedia Scenarios," IEEE MultiMedia, vol. 2, no. 3, Fall 1995, pp. 24-31.
[16] S.C. Kleene, Representation of Events in Nerve Nets and Finite Automata, Automata Studies. Princeton, N.J.: Princeton Univ. Press, pp. 3-41, 1956.
[17] C.C. Lin, J. Xiang, and S.K. Chang, “Transformation and Exchange of Multimedia Objects in Distributed Multimedia Systems,” Proc. ACM Multimedia Systems J., vol. 4, no. 1, Feb. 1996.
[18] T. Little and A. Ghafoor, "Synchronization and Storage for Multimedia Objects," IEEE J. Selected Areas in Comm., vol. 8, no. 3, pp. 413-427, Apr. 1990.
[19] T.D.C. Little and A. Ghafoor, “Interval-Based Conceptual Models for Time-Dependent Multimedia Data,” IEEE Trans. Knowledge and Data Eng., vol. 5, no. 4, pp. 551-563, Aug. 1993.
[20] E. Oomoto, “Design and Implementation of a Video-Object Database System,” IEEE Trans. Knowledge and Data Eng., vol. 5, no. 4, pp. 629-643, Aug. 1993.
[21] M.T. Özsu, D. Duane, G. El-Medani, and C. Vittal, “An Object-Oriented Multimedia Database System for a News-on-Demand Application,” ACM Multimedia Systems J., vol. 3, pp. 182-203, Nov. 1995.
[22] J.L. Peterson, “Petri Nets,” ACM Computing Surveys, vol. 9, no. 3, Sept. 1977.
[23] H. Thimm and W. Klas, "δ-Sets for Optimal Reactive Adaptive Playout Management in Distributed Multimedia Database Systems," Proc. 12th Int'l Conf. Data Eng., pp. 584-592, Feb. 1996.
[24] W.A. Woods, "Transition Network Grammars for Natural Language Analysis," Comm. ACM, vol. 13, no. 10, 1970, pp. 591-606.

Index Terms:
Augmented Transition Network (ATN), multimedia database systems, multimedia input string, multimedia presentations, semantic object.
Shu-Ching Chen, R.L. Kashyap, "A Spatio-Temporal Semantic Model for Multimedia Database Systems and Multimedia Information Systems," IEEE Transactions on Knowledge and Data Engineering, vol. 13, no. 4, pp. 607-622, July-Aug. 2001, doi:10.1109/69.940735
Usage of this product signifies your acceptance of the Terms of Use.