This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Querying Multimedia Presentations Based on Content
May/June 1999 (vol. 11 no. 3)
pp. 361-385

Abstract—In this paper, we consider the problem of querying multimedia presentations based on content information. We believe that presentations should become an integral part of multimedia database systems and users should be able to store, query, and, possibly, manipulate multimedia presentations using a single database management system software. Multimedia presentations are modeled as presentation graphs, which are directed acyclic graphs that visually specify multimedia presentations. Each node of a presentation graph represents a media stream. Edges depict sequential or concurrent playout of streams during the presentation. Information captured in each individual stream and the presentation order of streams constitute the content information of the presentation. Querying multimedia presentation graphs based on content is important for the retrieval of information from a database. We present a graph data model for the specification of multimedia presentations and discuss query languages as effective tools to query and manipulate multimedia presentation graphs with respect to content information. To query the information flow throughout a multimedia presentation, as well as in each individual multimedia stream, we use revised versions of temporal operators Next, Connected, and Until, together with path formulas. These constructs allow us to specify and query paths along a presentation graph. We present an icon-based, graphical query language, GVISUAL, that provides iconic representations for these constructs and a user-friendly graphical interface for query specification. We also present an OQL-like language, GOQL (Graph OQL), with similar constructs, that allows textual and more traditional specifications of graph queries. Finally, we introduce GCalculus (Graph Calculus), a calculus-based language that establishes the formal grounds for the use of temporal operators in path formulas and for querying presentation graphs with respect to content information. We also discuss GCalculus/S (GCalculus with sets) which avoids highly complex query expressions by eliminating universal path quantifier, the negation operator, and the universal quantifier. GCalculus/S represents the formal basis for GVISUAL, i.e., GVISUAL uses the constructs of GCalculus/S directly.

[1] A. Al-Hamdani, "Implementing GVISUAL for Multimedia Presentation Querying," MS thesis, Case Western Reserve Univ., June 1998.
[2] S. Adali, K.S. Candan, S.-S. Chen, K. Erol, and V.S. Subrahmanian, “Advanced Video Information Systems,” ACM Multimedia Systems J., vol. 4, pp. 172-186, 1996.
[3] Apple Computer, Macintosh Human Interface Guidelines.Reading, Mass.: Addison-Wesley, 1992.
[4] N.H. Balkir, "VISUAL," MS thesis, Computer Eng. and Science Dept., Case Western Reserve Univ., May 1995.
[5] N.H. Balkir and G. Özsoyoglu, "Multimedia Presentation Servers: Buffer Management and Admission Control," Proc. 1998 IEEE Int'l Workshop Multimedia DBMSs,Dayton, Ohio, Aug. 1998.
[6] T. Bozkaya, N.H. Balkir, and T. Lee, "Efficient Evaluation of Path Algebra Expressions," technical report, Case Western Reserve Univ., June 1997.
[7] N.H. Balkir, Y. Lin, N. Yazdani, G. Özsoyoglu, and Z.M. Özsoyoglu, "ViSiOn: VISUAL, VStore, and O-Algebra," unpublished manuscript, 1997.
[8] G. Beekman, "Multimedia Authoring Software," http://csdl.computer.org/comp/trans/tk/1993/ 04/k0564abs.htmhttp://www.awl.com/he/ is/bclink/bclink2Authoring.html.
[9] G. Blakowski and R. Steinmetz,"A Media Synchronization Survey: Reference Model, Specification, and Case Studies," IEEE J. Selected Areas in Comm., vol. 14, no. 1, Jan. 1996, pp. 5-35.
[10] N.H. Balkir, G. Özsoyoglu, and Z.M Özsoyoglu, "A Graphical Query Language: VISUAL," submitted for journal publication, 1997.
[11] N.H. Balkir, E. Sukan, G. Özsoyoglu, and Z.M Özsoyoglu, "VISUAL: A Graphical Icon-Based Query Language," Proc. IEEE ICDE Conf., pp. 524-533, 1996.
[12] R. Cai, "Implementing Path Algebra Operators for the VStore System," MS project in progress, Case Western Reserve Univ., 1998.
[13] J. Carroll and D. Long, Theory of Finite Automata with an Introduction to Formal Languages. Prentice Hall, 1989.
[14] The Object Database Standard: ODMG-93, Release 1.2, R.G.G. Cattell, ed., 1996.
[15] M.P. Consens and A.O. Mendelzon,“GraphLog: A visual formalism for real life recursion,” Proc. Ninth ACM Symp. Principles of Database Systems, pp. 404-416, Apr. 1990.
[16] M. Consens and A. Mendelzon, "Hy+: A Hygraph-Based Query and Visualization System," Proc. ACM SIGMOD Conf., 1993.
[17] I.F. Cruz, A. Mendelzon, and P.T. Wood, "A Graphical Query Language Supporting Recursion," Proc. ACM SIGMOD Conf., 1987.
[18] I.F. Cruz, A. Mendelzon, and P.T. Wood, "G+: Recursive Queries without Recursion," Proc. Second Int'l Conf. Expert Database Systems, 1988.
[19] I. Cruz, “Doodle: A Visual Language for Object-Oriented Databases,” Proc. ACM SIGMOD Conf., 1992.
[20] Y.F. Day, S. Dagtas, M. Iino, A. Khokhar, and A. Ghafoor, “Object-Oriented Conceptual Modeling of Video Data,” Proc. Data Eng. (DE '95), pp. 401-408, 1995.
[21] E. Emerson, "Temporal and Modal Logic," Handbook of Theoretical Computer Science, J. Leeuwen, ed., chapter 16, pp. 995-1, 072. Elsevier, 1990.
[22] J. Gao, "OAOpt: An Optimizer's Design and Implementation for Object-Oriented Databases," MS project, Case Western Reserve Univ., Feb. 1997.
[23] M. Gyssens, J. Paredaens, J. Van den Bussche, and D. Van Gucht, "A Graph-Oriented Object Database Model," IEEE Trans. Knowledge and Data Eng., vol. 6, no. 4, pp. 572-586, 1994.
[24] D. Gentner and J. Nielson, "The Anti-Mac Interface," Comm. ACM, vol. 39, no. 8, pp. 70-82, Aug. 1996.
[25] M. Golebiewski, "Extending VISUAL with Drop-and-Drag Features," MS project, Case Western Reserve Univ., May 1996.
[26] M. Gyssens,J. Paredaens,, and D. Van Gucht,“A graph-oriented object database model,” Proc. Ninth ACM Symp. Principles of Database Systems, pp. 417-424, Apr. 1990.
[27] M. Gyssens, J. Paradaens, and D. Van Gucht, "A Graph-Oriented Object Model for Database End-User Interfaces," Proc. ACM SIGMOD Conf., 1990.
[28] R. Guting, "GraphDB: Modeling and Querying Graphs in Databases," Proc. Int'l Conf. Very Large Data Bases, pp. 297-308, 1994.
[29] M. Haindl, "A New Multimedia Synchronization Model," IEEE J. Selected Areas in Comm., vol. 14, no. 1, Jan. 1996.
[30] V. Hakkoymaz, "A Constraint-Driven Methodology for Designing a Multimedia Presentation System from Multimedia Databases," PhD thesis, Computer Eng. and Science Dept., Case Western Reserve Univ., Dec. 1996.
[31] V. Hakkoymaz and G. Özsoyoglu, "Constrain-Based Automation of Multimedia Presentation Assembly," ACM Multimedia Systems J., to appear 1998.
[32] E. Hwang and V.S. Subrahmanian, "Querying Video Libraries," technical report, Univ. of Maryland, College Park, June 1995.
[33] P. Kanellakis, C. Lecluse, and P. Richard, "Introduction to the Data Model," Building an Object-Oriented Database System: The Story of O2, pp. 61-76, Morgan-Kaufmann, 1992.
[34] T. Lee and G. Özsoyoglu, "On Presentation Languages for Multimedia Databases," Case Western Reserve Univ. technical report, 1996.
[35] T. Lee and G. Özsoyoglu, "Query Processing Techniques for Multimedia Presentations," Proc. IEEE RIDE Conf., Feb. 1998.
[36] T. Lee, L. Sheng, N.H. Balkir, A. Al-Hamdani, G. Özsoyoglu, and Z.M. Özsoyoglu, "Query Processing Techniques for Multimedia Presentations," submitted for journal publication, Apr. 1998.
[37] O. Lichtenstein, A. Pneuli, and L. Zuck, "The Glory of the Past," Lecture Notes in Computer Science, vol. 193. Springer-Verlag, 1985.
[38] T. Little and A. Ghafoor, "Synchronization and Storage for Multimedia Objects," IEEE J. Selected Areas in Comm., vol. 8, no. 3, pp. 413-427, Apr. 1990.
[39] J. Lin and Z.M. Ozsoyoglu, “Processing OODB Queries by O-Algebra,” Proc. Eighth Int'l Conf. Information and Knowledge Management, pp. 134-142, Nov. 1996.
[40] J. Lin, "Processing OODB Queries by O-Algebra," PhD thesis, Case Western Reserve Univ., May 1998.
[41] C. Lecluse and P. Richard, "Modeling Complex Structures in Object-Oriented Databases," Proc. Symp. Principles of Database Systems, pp. 360-368, 1989.
[42] K.L. Liu, A.P. Sistla, C. Yu, and N. Rishe, "Query Processing in a Video Retrieval System," Proc. IEEE ICDE Conf., 1998.
[43] J. Lin, X. Zhang, and Z.M. Özsoyoglu, "A Complete Solution of Reducing OODB Nested Queries," submitted for publication, 1997.
[44] S. Marcus and V.S. Subrahmanian, “Foundations of Multimedia Database Systems,” J. ACM, vol. 43, no. 3, pp. 474-523, 1996.
[45] M. Magel, "Comparative Review of Authoring Tools," http://www.allencomm.com/p&s/software/ quest/whtpgsquwhite.html.
[46] A. Oeftering, "Picking a Multimedia Authoring Tool," http://www.datatech.com/hots96_3.htm.
[47] G. Ozsoyoglu, V. Matos, and Z.M. Ozsoyoglu, “Query Processing Techniques in the Summary-Table-by-Example Database Query Language,” Proc. ACM Trans. Database Systems, vol. 14, no. 4, Dec. 1989.
[48] R. Ortega, "Design and Implementation of a Video Presentation Tool," MS project, Case Western Reserve Univ., May 1997.
[49] G. Özsoyoglu and W. Abdul-Qader, "Human Factors Study of Two Screen-Oriented Query Languages: STBE and QBE," Information and Technology J., vol. 34, no. 1, pp. 3-15, Jan. 1992.
[50] G. Ozsoyoglu and H. Wang, “A Relational Calculus with Set Operators, Its Safety, and Equivalent Graphical Languages,” IEEE Trans. Software Eng., vol. 15, no. 9, Sept. 1989.
[51] Products: Director, Authorware (Company: Macromedia)http://wwwmacromedia.com, IconAuthor (AIMTech) http://www.aimtech.com, Quest (Allen Communications)http:/www.allencomm.com, Astound (Gold Disk)http:/www.golddisk.com, Hypercard (Apple)http:/www.hypercard.apple.com.
[52] B. Ramasamy, "Implementation of O-Algebra: An Algebra for OO Databases," MS project, Case Western Reserve Univ., Apr. 1996.
[53] P. Reisner, "Use of Psychological Experimentation as an Aid to Development of a Query Language," IEEE Trans. Software Eng., pp. 218-229, Mar. 1977.
[54] K. Renganathan, "A Stand-Alone Multimedia Authoring Tool," MS project, Case Western Reserve Univ., Oct. 1997.
[55] J. Richardson, "Supporting Lists in a Data Model (A Timely Approach)," Proc. VLDB Conf., 1992.
[56] B. Shneiderman, "Improving the Human Factors Aspect of Database Interactions," ACM Trans. Database Systems, vol. 3, no. 3, pp. 417-439, 1978.
[57] L. Sheng and Z.M. Özsoyoglu, "Querying Multimedia Presentation Graphs," Case Western Reserve Univ. technical report, 1997.
[58] R. Steinmetz,"Synchronization Properties in Multimedia Systems," IEEE J. Selected Areas in Comm., vol. 8, no. 3, Apr. 1990, pp. 401-412.
[59] V.S. Subrahmanian, Principles of Multimedia Database Systems. Morgan Kaufmann, 1998.
[60] S. Subhani, "Implementing O-Algebra Operators in VStore," MS project in progress, Case Western Reserve Univ., 1998.
[61] A.P. Sistla, C. Yu, and R. Venkatasubrahmanian, "Similarity Based Retrieval of Videos," Proc. IEEE Data Eng. Conf., 1997.
[62] J.C. Thomas and J.D. Gould, "A Psychological Study of Query by Example," Proc. AFIPS Nat'l Conf., 1975.
[63] R. Touma, "ViSiOn Database Browser and Administration Server Upgrade," MS project, Case Western Reserve Univ., Nov. 1997.
[64] P. Wolper, "Temporal Logic Can Be More Expressive," Proc. IEEE Symp. Foundations of Computer Science, pp. 340-348, 1981.
[65] M.M. Zloof, "Query-by-Example: A Database Language," IBM Systems J., vol. 21, no. 3, 1977.
[66] H. Yao, "Database Browser and Administration Server for VISUAL," MS project, Case Western Reserve Univ., June 1997.

Index Terms:
Multimedia databases, graphical query languages, multimedia presentation.
Citation:
Taekyong Lee, Lei Sheng, Tolga Bozkaya, Nevzat Hurkan Balkir, Z. Meral Özsoyoglu, Gultekin Özsoyoglu, "Querying Multimedia Presentations Based on Content," IEEE Transactions on Knowledge and Data Engineering, vol. 11, no. 3, pp. 361-385, May-June 1999, doi:10.1109/69.774099
Usage of this product signifies your acceptance of the Terms of Use.