The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - Oct. (2013 vol.19)
pp: 1664-1676
Yongwei Nie , Comput. Sch., Wuhan Univ., Wuhan, China
Chunxia Xiao , Comput. Sch., Wuhan Univ., Wuhan, China
Hanqiu Sun , Dept. of Comput. Sci. & Eng., Chinese Univ. of Hong Kong, Shatin, China
Ping Li , Dept. of Comput. Sci. & Eng., Chinese Univ. of Hong Kong, Shatin, China
ABSTRACT
Video synopsis aims at providing condensed representations of video data sets that can be easily captured from digital cameras nowadays, especially for daily surveillance videos. Previous work in video synopsis usually moves active objects along the time axis, which inevitably causes collisions among the moving objects if compressed much. In this paper, we propose a novel approach for compact video synopsis using a unified spatiotemporal optimization. Our approach globally shifts moving objects in both spatial and temporal domains, which shifting objects temporally to reduce the length of the video and shifting colliding objects spatially to avoid visible collision artifacts. Furthermore, using a multilevel patch relocation (MPR) method, the moving space of the original video is expanded into a compact background based on environmental content to fit with the shifted objects. The shifted objects are finally composited with the expanded moving space to obtain the high-quality video synopsis, which is more condensed while remaining free of collision artifacts. Our experimental results have shown that the compact video synopsis we produced can be browsed quickly, preserves relative spatiotemporal relationships, and avoids motion collisions.
INDEX TERMS
Spatiotemporal phenomena, Surveillance, Optimization, Visualization, Space vehicles, Context, Trajectory,patch relocation, Video synopsis, surveillance, optimization
CITATION
Yongwei Nie, Chunxia Xiao, Hanqiu Sun, Ping Li, "Compact Video Synopsis via Global Spatiotemporal Optimization", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 10, pp. 1664-1676, Oct. 2013, doi:10.1109/TVCG.2012.176
REFERENCES
[1] Y. Li, T. Zhang, and D. Tretter, "An Overview of Video Abstraction Techniques," HP Laboratories Palo Alto, 2001.
[2] H. Kang, X. Chen, Y. Matsushita, and X. Tang, "Space-Time Video Montage," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 1331-1338, 2006.
[3] B. Truong and S. Venkatesh, "Video Abstraction: A Systematic Review and Classification," ACM Trans. Multimedia Computing, Comm., and Applications (TOMCCAP), vol. 3, no. 1,article 3, 2007.
[4] Y. Pritch, A. Rav-Acha, and S. Peleg, "Nonchronological Video Synopsis and Indexing," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 11, pp. 1971-1987, Nov. 2008.
[5] D. Simakov, Y. Caspi, E. Shechtman, and M. Irani, "Summarizing Visual Data Using Bidirectional Similarity," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '08), pp. 1-8, 2008.
[6] C. Kim and J. Hwang, "An Integrated Scheme for Object-Based Video Abstraction," Proc. ACM Eighth Int'l Conf. Multimedia, pp. 303-311, 2000.
[7] J. Nam and A. Tewfik, "Video Abstract of Video," Proc. IEEE Third Workshop Multimedia Signal Processing, pp. 117-122, 1999.
[8] E. Bennett and L. McMillan, "Computational Time-Lapse Video," ACM Trans. Graphics (TOG), vol. 26, no. 3,article 102, 2007.
[9] A. Rav-Acha, Y. Pritch, and S. Peleg, "Making a Long Video Short: Dynamic Video Synopsis," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 435-441, 2006.
[10] Y. Pritch, A. Rav-Acha, A. Gutman, and S. Peleg, "Webcam Synopsis: Peeking around the World," Proc. IEEE 11th Int'l Conf. Computer Vision (ICCV '07), pp. 1-8, 2007.
[11] T. Liu, X. Zhang, J. Feng, and K. Lo, "Shot Reconstruction Degree: A Novel Criterion for Key Frame Selection," Pattern Recognition Letters, vol. 25, no. 12, pp. 1451-1457, 2004.
[12] X. Zhu, X. Wu, J. Fan, A. Elmagarmid, and W. Aref, "Exploring Video Content Structure for Hierarchical Summarization," Multimedia Systems, vol. 10, no. 2, pp. 98-115, 2004.
[13] N. Petrovic, N. Jojic, and T. Huang, "Adaptive Video Fast Forward," Multimedia Tools and Applications, vol. 26, no. 3, pp. 327-344, 2005.
[14] C. Taskiran, Z. Pizlo, A. Amir, D. Ponceleon, and E. Delp, "Automated Video Program Summarization Using Speech Transcripts," IEEE Trans. Multimedia, vol. 8, no. 4, pp. 775-791, Aug. 2006.
[15] Y. Li, S. Narayanan, and C. Kuo, Movie Content Analysis, Indexing and Skimming via Multimodal Information. Kluwer Academic, 2003.
[16] J. Wu, Perspectives on Content-Based Multimedia Systems. Springer, 2000.
[17] J. Ouyang, J. Li, and Y. Zhang, "Replay Boundary Detection in Mpeg Compressed Video," Proc. Int'l Conf. Machine Learning and Cybernetics, vol. 5, pp. 2800-2804, 2003.
[18] C. Barnes, E. Shechtman, A. Finkelstein, and D. Goldman, "Patch-Match: A Randomized Correspondence Algorithm for Structural Image Editing," ACM Trans. Graphics (TOG), vol. 28, no. 3,article 24, 2009.
[19] C. Xiao, M. Liu, Y. Nie, and Z. Dong, "Fast Exact Nearest Patch Match for Patch-Based Image Editing and Processing," IEEE Trans. Visualization and Computer Graphics, vol. 17, no. 8 pp. 1122-1134, Aug. 2011.
[20] V. Kwatra, I. Essa, A. Bobick, and N. Kwatra, "Texture Optimization for Example-Based Synthesis," ACM Trans. Graphics (TOG), vol. 24, no. 3, pp. 795-802, 2005.
[21] L. Wei and M. Levoy, "Fast Texture Synthesis Using Tree-Structured Vector Quantization," Proc. 27th Ann. Conf. Computer Graphics and Interactives Techniques, pp. 479-488, 2000.
[22] C. Xiao, Y. Nie, W. Hua, and W. Zheng, "Fast Multi-Scale Joint Bilateral Texture Upsampling," The Visual Computer, vol. 26, no. 4, pp. 263-275, 2010.
[23] Y. Wexler, E. Shechtman, and M. Irani, "Space-Time Video Completion," Proc. IEEE Computer Soc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. I-120-I-127 Vol.1, 2004.
[24] J. Sun, L. Yuan, J. Jia, and H. Shum, "Image Completion with Structure Propagation," ACM Trans. Graphics (ToG), vol. 24, no. 3, pp. 861-868, 2005.
[25] C. Xiao, S. Liu, H. Fu, C. Lin, C. Song, Z. Huang, F. He, and Q. Peng, "Video Completion and Synthesis," Computer Animation and Virtual Worlds, vol. 19, nos. 3/4, pp. 341-353, 2008.
[26] T. Cho, M. Butman, S. Avidan, and W. Freeman, "The Patch Transform and Its Applications to Image Editing," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2008.
[27] T. Cho, S. Avidan, and W. Freeman, "The Patch Transform," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 8, pp. 1489-1501, Aug. 2010.
[28] A. Jain, R. Duin, and J. Mao, "Statistical Pattern Recognition: A Review," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 4-37, Jan. 2000.
[29] S. Cohen, "Background Estimation As a Labeling Problem," Proc. IEEE 10th Int'l Conf. Computer Vision (ICCV '05), vol. 2, pp. 1034-1041, 2005.
[30] P. Pérez, M. Gangnet, and A. Blake, "Poisson Image Editing," ACM Trans. Graphics (TOG), vol. 22, no. 3, pp. 313-318, 2003.
[31] J. Sun, W. Zhang, X. Tang, and H. Shum, "Background Cut," Proc. European Conf. Computer Vision (ECCV '06), pp. 628-641, 2006.
[32] P. KaewTraKulPong and R. Bowden, "An Improved Adaptive Background Mixture Model for Real-Time Tracking with Shadow Detection," Proc. Second European Workshop Advanced Video Based Surveillance Systems, vol. 25, 2001.
[33] V. Kolmogorov and R. Zabih, "What Energy Functions Can be Minimized via Graph Cuts?" IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 147-159, Feb. 2004.
[34] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[35] J. Yedidia, W. Freeman, and Y. Weiss, "Understanding Belief Propagation and Its Generalizations," Exploring Artificial Intelligence in the New Millennium, vol. 8, pp. 236-239, 2003.
[36] S. Avidan and A. Shamir, "Seam Carving for Content-Aware Image Resizing," ACM Trans. Graphics (TOG), vol 26, no. 3,article 10, 2007.
168 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool