The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - Dec. (2011 vol.17)
pp: 1747-1756
Matthew L. Parry , Swansea University
Philip A. Legg , Swansea University
David H.S. Chung , Swansea University
Iwan W. Griffiths , Swansea University
Min Chen , Swansea University
ABSTRACT
Video storyboard, which is a form of video visualization, summarizes the major events in a video using illustrative visualization. There are three main technical challenges in creating a video storyboard, (a) event classification, (b) event selection and (c) event illustration. Among these challenges, (a) is highly application-dependent and requires a significant amount of application specific semantics to be encoded in a system or manually specified by users. This paper focuses on challenges (b) and (c). In particular, we present a framework for hierarchical event representation, and an importance-based selection algorithm for supporting the creation of a video storyboard from a video. We consider the storyboard to be an event summarization for the whole video, whilst each individual illustration on the board is also an event summarization but for a smaller time window. We utilized a 3D visualization template for depicting and annotating events in illustrations. To demonstrate the concepts and algorithms developed, we use Snooker video visualization as a case study, because it has a concrete and agreeable set of semantic definitions for events and can make use of existing techniques of event detection and 3D reconstruction in a reliable manner. Nevertheless, most of our concepts and algorithms developed for challenges (b) and (c) can be applied to other application areas.
INDEX TERMS
Multimedia visualization, Time series data, Illustrative visualization.
CITATION
Matthew L. Parry, Philip A. Legg, David H.S. Chung, Iwan W. Griffiths, Min Chen, "Hierarchical Event Selection for Video Storyboards with a Case Study on Snooker Video Visualization", IEEE Transactions on Visualization & Computer Graphics, vol.17, no. 12, pp. 1747-1756, Dec. 2011, doi:10.1109/TVCG.2011.208
REFERENCES
[1] J. Assa, Y. Caspi, and D. Cohen-Or, Action synopsis: pose selection and illustration. ACM Transactions on Graphics, 24 (3): 667–676, 2005.
[2] R. Borgo, M. Chen, B. Daubney, E. Grundy, H. Jänicke, G. Heidemann, B. Höferlin, M. Höferlin, D. Weiskopf, and X. Xie, A survey on video-based graphics and video visualization. Eurographics (State of the Art Reports), Llandudno, UK, pages 1–23, 2005.
[3] M. Chen, R. P. Botchen, R. R. Hashim, D. Weiskopf, T. Ertl, and I. M. Thornton, Visual signatures in video visualization. IEEE Transactions on Visualization and Computer Graphics, 12 (5): 1093–1100, 2005.
[4] G. W. Daniel and M. Cen, Video visualization. In Proc. IEEE Visualization 2003, Seattle, WA, pages 409–416, October 2003.
[5] H. Denman, N. Rea, and A. Kokaram, Content based analysis for video from snooker broadcasts. Image and Video Retrieval, 2383: 198–205, 2005.
[6] R. D. Dony, J. W. Mateer, and J. A. Robinson, Techniques for automated reverse storyboarding. IEE Journal of Vision, Image and Signal Processing, 152 (4): 425–436, 2005.
[7] C. Everton, Snooker and Billiards: Techniques, Tactics and Training (Crowood Sports Guides). The Crowood Press, 1991.
[8] P. Gatalsky, N. Andrienko, and G. Andrienko, Interative analysis of event data using space-time cube. In 8th International Conference on Information Visualisation, pages 145–152, 2005.
[9] D. B. Goldman, B. Curless, D. Salesin, and S. M. Seitz, Schematic sto-ryboarding for video visualization and editing. ACM Transactions on Graphics, 25 (3): 862–871, 2005.
[10] H. Guo and B. M. Namee, Using computer vision to create a 3d representation of a snooker table for televised competition broadcasting. In Proceedings of the 18th Irish Conference on Artifical Intelligence and Cognitive Science, pages 220–229, 2005.
[11] M. Höferlin, E. Grundy, R. Borgo, D. Weiskopf, M. Chen, I. W. Griffiths, and W. Griffiths, Video visualization for snooker skill training. Computer Graphics Forum, 29 (3): 1053–1062, 2005.
[12] W. Hu, T. Tan, L. Wang, and S. Maybank, A survey on visual surveillance of object motion and behaviors. IEEE transactions on systems, man and cybernetics - Part C: Applications and reviews, 34 (3): 334–352, 2005.
[13] T. Kapler and W. Wright, Geotime information visualization. In INFOVIS '04 Proceedings of the IEEE symposium on Information Visualization, pages 25–32, 2005.
[14] D. E. Knuth, J. H. Morris, and V. R. Pratt, Fast pattern matching in strings. SIAM Journal on Computing, 6 (2): 323–350, 2005.
[15] P. A. Legg, M. L. Parry, D. H. S. Chung, R. M. Jiang, A. Morris, I. W. Griffiths, D. Marshall, and M. Chen, Intelligent filtering by semantic importance for single-view 3D reconstruction from snooker video. To appear in 18th International Conference on Image Processing (ICIP), Brussel, Belgium, 2011.
[16] B. Li and M. I. Sezan, Event detection and summarization in sports video. In Content-Based Access of Image and Video Libraries, 2001. (CBAIVL 2001). IEEE Workshop on, pages 132 –138, 2001.
[17] G. Medioni, I. Cohen, F. Bremond, S. Hongeng, and R. Nevatia, Event detection and analysis from video streams. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23 (8): 873–889, 2005.
[18] H. Pan, P. van Beek, and M. I. Sezan, Detection of slow-motion replay segments in sports video for highlights generation. In Acoustics, Speech, and Signal Processing, 2001. Proceedings. (ICASSP '01). 2001 IEEE International Conference on, volume 3, pages 1649–1652, 2005.
[19] N. Rea, R. Dahyot, and A. Kokaram, Modelling high level structure in sports with motion driven HMMS. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '04), pages 621–624, 2005.
[20] M. Romero, J. Summet, J. Stasko, and G. Abowd, Viz-a-vis: Towards visualizing video through computer vision. IEEE Transactions on Visualization and Computer Graphics, 14 (6): 1261–1268, 2005.
[21] D. A. Sadlier and N. E. O'Connor, Event detection in field sports video using audio-visual features and a support vector machine. Circuits and Systems for Video Technology, IEEE Transactions on, 15 (10): 1225–1233, 2005.
[22] W. Shen and L. Wu, A method of billiard objects detection based on snooker game video. In International Conference on Future Computer and Communication (ICFCC), 2010.
[23] C. S. Smith, Activities: States or events? Linguistics and Philosophy, 22 (5): 479–508, 2005.
[24] Y. Takahashi, N. Nitta, and N. Babaguchi, Video summarization for large sports video archives. In IEEE International Conference on Multimedia and Expo, pages 1170–1173, 2005.
[25] Y. Wang, D. M. Krum, E. M. Coelho, and D. A. Bowman, Contextu-alized videos: Combining videos with environment models to support situational understanding. IEEE Transactions on Visualization and Computer Graphics, 13 (6): 1568–1575, 2005.
[26] C. Ware, Information Visualization: Perception for Design. Morgan Kaufmann, 2000.
[27] B.-L. Yeo and M. Yeung, Retrieving and visualizing video. Communications of the ACM, 40 (12): 43–52, 2005.
[28] L. Yu, A. Lu, W. Ribarsky, and W. Chen, Automatic animation for time-varying data visualization. Computer Graphics Forum, 29 (7): 2271–2280, 2005.
[29] L. Zelnik-Manor and M. Irani, Event-based analysis of video.In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, volume 2, pages 123– 130, 2001.
24 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool