This Article 
 Bibliographic References 
 Add to: 
Hierarchical Event Selection for Video Storyboards with a Case Study on Snooker Video Visualization
Dec. 2011 (vol. 17 no. 12)
pp. 1747-1756
Matthew L. Parry, Swansea University
Philip A. Legg, Swansea University
David H.S. Chung, Swansea University
Iwan W. Griffiths, Swansea University
Min Chen, Swansea University
Video storyboard, which is a form of video visualization, summarizes the major events in a video using illustrative visualization. There are three main technical challenges in creating a video storyboard, (a) event classification, (b) event selection and (c) event illustration. Among these challenges, (a) is highly application-dependent and requires a significant amount of application specific semantics to be encoded in a system or manually specified by users. This paper focuses on challenges (b) and (c). In particular, we present a framework for hierarchical event representation, and an importance-based selection algorithm for supporting the creation of a video storyboard from a video. We consider the storyboard to be an event summarization for the whole video, whilst each individual illustration on the board is also an event summarization but for a smaller time window. We utilized a 3D visualization template for depicting and annotating events in illustrations. To demonstrate the concepts and algorithms developed, we use Snooker video visualization as a case study, because it has a concrete and agreeable set of semantic definitions for events and can make use of existing techniques of event detection and 3D reconstruction in a reliable manner. Nevertheless, most of our concepts and algorithms developed for challenges (b) and (c) can be applied to other application areas.

[1] J. Assa, Y. Caspi, and D. Cohen-Or, Action synopsis: pose selection and illustration. ACM Transactions on Graphics, 24 (3): 667–676, 2005.
[2] R. Borgo, M. Chen, B. Daubney, E. Grundy, H. Jänicke, G. Heidemann, B. Höferlin, M. Höferlin, D. Weiskopf, and X. Xie, A survey on video-based graphics and video visualization. Eurographics (State of the Art Reports), Llandudno, UK, pages 1–23, 2005.
[3] M. Chen, R. P. Botchen, R. R. Hashim, D. Weiskopf, T. Ertl, and I. M. Thornton, Visual signatures in video visualization. IEEE Transactions on Visualization and Computer Graphics, 12 (5): 1093–1100, 2005.
[4] G. W. Daniel and M. Cen, Video visualization. In Proc. IEEE Visualization 2003, Seattle, WA, pages 409–416, October 2003.
[5] H. Denman, N. Rea, and A. Kokaram, Content based analysis for video from snooker broadcasts. Image and Video Retrieval, 2383: 198–205, 2005.
[6] R. D. Dony, J. W. Mateer, and J. A. Robinson, Techniques for automated reverse storyboarding. IEE Journal of Vision, Image and Signal Processing, 152 (4): 425–436, 2005.
[7] C. Everton, Snooker and Billiards: Techniques, Tactics and Training (Crowood Sports Guides). The Crowood Press, 1991.
[8] P. Gatalsky, N. Andrienko, and G. Andrienko, Interative analysis of event data using space-time cube. In 8th International Conference on Information Visualisation, pages 145–152, 2005.
[9] D. B. Goldman, B. Curless, D. Salesin, and S. M. Seitz, Schematic sto-ryboarding for video visualization and editing. ACM Transactions on Graphics, 25 (3): 862–871, 2005.
[10] H. Guo and B. M. Namee, Using computer vision to create a 3d representation of a snooker table for televised competition broadcasting. In Proceedings of the 18th Irish Conference on Artifical Intelligence and Cognitive Science, pages 220–229, 2005.
[11] M. Höferlin, E. Grundy, R. Borgo, D. Weiskopf, M. Chen, I. W. Griffiths, and W. Griffiths, Video visualization for snooker skill training. Computer Graphics Forum, 29 (3): 1053–1062, 2005.
[12] W. Hu, T. Tan, L. Wang, and S. Maybank, A survey on visual surveillance of object motion and behaviors. IEEE transactions on systems, man and cybernetics - Part C: Applications and reviews, 34 (3): 334–352, 2005.
[13] T. Kapler and W. Wright, Geotime information visualization. In INFOVIS '04 Proceedings of the IEEE symposium on Information Visualization, pages 25–32, 2005.
[14] D. E. Knuth, J. H. Morris, and V. R. Pratt, Fast pattern matching in strings. SIAM Journal on Computing, 6 (2): 323–350, 2005.
[15] P. A. Legg, M. L. Parry, D. H. S. Chung, R. M. Jiang, A. Morris, I. W. Griffiths, D. Marshall, and M. Chen, Intelligent filtering by semantic importance for single-view 3D reconstruction from snooker video. To appear in 18th International Conference on Image Processing (ICIP), Brussel, Belgium, 2011.
[16] B. Li and M. I. Sezan, Event detection and summarization in sports video. In Content-Based Access of Image and Video Libraries, 2001. (CBAIVL 2001). IEEE Workshop on, pages 132 –138, 2001.
[17] G. Medioni, I. Cohen, F. Bremond, S. Hongeng, and R. Nevatia, Event detection and analysis from video streams. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23 (8): 873–889, 2005.
[18] H. Pan, P. van Beek, and M. I. Sezan, Detection of slow-motion replay segments in sports video for highlights generation. In Acoustics, Speech, and Signal Processing, 2001. Proceedings. (ICASSP '01). 2001 IEEE International Conference on, volume 3, pages 1649–1652, 2005.
[19] N. Rea, R. Dahyot, and A. Kokaram, Modelling high level structure in sports with motion driven HMMS. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '04), pages 621–624, 2005.
[20] M. Romero, J. Summet, J. Stasko, and G. Abowd, Viz-a-vis: Towards visualizing video through computer vision. IEEE Transactions on Visualization and Computer Graphics, 14 (6): 1261–1268, 2005.
[21] D. A. Sadlier and N. E. O'Connor, Event detection in field sports video using audio-visual features and a support vector machine. Circuits and Systems for Video Technology, IEEE Transactions on, 15 (10): 1225–1233, 2005.
[22] W. Shen and L. Wu, A method of billiard objects detection based on snooker game video. In International Conference on Future Computer and Communication (ICFCC), 2010.
[23] C. S. Smith, Activities: States or events? Linguistics and Philosophy, 22 (5): 479–508, 2005.
[24] Y. Takahashi, N. Nitta, and N. Babaguchi, Video summarization for large sports video archives. In IEEE International Conference on Multimedia and Expo, pages 1170–1173, 2005.
[25] Y. Wang, D. M. Krum, E. M. Coelho, and D. A. Bowman, Contextu-alized videos: Combining videos with environment models to support situational understanding. IEEE Transactions on Visualization and Computer Graphics, 13 (6): 1568–1575, 2005.
[26] C. Ware, Information Visualization: Perception for Design. Morgan Kaufmann, 2000.
[27] B.-L. Yeo and M. Yeung, Retrieving and visualizing video. Communications of the ACM, 40 (12): 43–52, 2005.
[28] L. Yu, A. Lu, W. Ribarsky, and W. Chen, Automatic animation for time-varying data visualization. Computer Graphics Forum, 29 (7): 2271–2280, 2005.
[29] L. Zelnik-Manor and M. Irani, Event-based analysis of video.In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, volume 2, pages 123– 130, 2001.

Index Terms:
Multimedia visualization, Time series data, Illustrative visualization.
Matthew L. Parry, Philip A. Legg, David H.S. Chung, Iwan W. Griffiths, Min Chen, "Hierarchical Event Selection for Video Storyboards with a Case Study on Snooker Video Visualization," IEEE Transactions on Visualization and Computer Graphics, vol. 17, no. 12, pp. 1747-1756, Dec. 2011, doi:10.1109/TVCG.2011.208
Usage of this product signifies your acceptance of the Terms of Use.