| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
Probabilistic Space-Time Video Modeling via Piecewise GMM
March 2004 (vol. 26 no. 3)
pp. 384-396
Abstract—In this paper, we describe a statistical video representation and modeling scheme. Video representation schemes are needed to segment a video stream into meaningful video-objects, useful for later indexing and retrieval applications. In the proposed methodology, unsupervised clustering via Gaussian mixture modeling extracts coherent space-time regions in feature space, and corresponding coherent segments (video-regions) in the video content. A key feature of the system is the analysis of video input as a single entity as opposed to a sequence of separate frames. Space and time are treated uniformly. The probabilistic space-time video representation scheme is extended to a piecewise GMM framework in which a succession of GMMs are extracted for the video sequence, instead of a single global model for the entire sequence. The piecewise GMM framework allows for the analysis of extended video sequences and the description of nonlinear, nonconvex motion patterns. The extracted space-time regions allow for the detection and recognition of video events. Results of segmenting video content into static versus dynamic video regions and video content editing are presented.
[1] 384 G. Ahanger and T.D.C. Little, A Survey of Technologies for Parsing and Indexing Digital Video J. Visual Comm. and Image Representation, vol. 7, no. 1, pp. 28-43, 1996.[2] C. Carson, S. Belongie, H. Greenspan, and J. Malik, Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 8, pp. 1026-1038, Aug. 2002.[3] J.R. Bergen, P.J. Burt, R. Hingorani, and S. Peleg, "A Three-Frame Algorithm for Estimating Two-Component Image Motion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, pp. 886-895, Sept. 1992.[4] C. Bregler, “Learning and Recognizing Human Dynamics in Video Sequences,” Proc. Conf. Computer Vision and Pattern Recognition, 1997.[5] R. Castagno, T. Ebrahimi, and M. Kunt, Video Segmentation Based on Multiple Features for Interactive Multimedia Applications IEEE Trans. Circuits and Systems for Video Technology, vol. 8, no. 5, pp. 562-571, 1998.[6] S.-F. Chang, W. Chen, H. Meng, H. Sundaram, and D. Zhong, A Fully Automated Content-Based Video Search Engine Supporting Spatiotemporal Queries IEEE Trans. Circuits and Systems for Video Technology, vol. 8, no. 5, pp. 602-615, 1998.[7] T.M. Cover and J.A. Thomas, Elements of Information Theory. John Wiley and Sons 1991.[8] D. DeMenthon, Spatio-Temporal Segmentation of Video by Hierarchical Mean Shift Analysis Proc. Statistical Methods in Video Processing Workshop (SMVP), June 2002.[9] A. Dempster, N. Laird, and D. Rubin, Maximum Likelihood From Incomplete Data via the em Algorithm J. Royal Statistical Soc. B, vol. 39, no. 1, pp. 1-38, 1997.[10] Y. Deng and B.S. Manjunath, Content-Based Search of Video Using Color, Texture and Motion Proc. Fourth IEEE Int'l Conf. Image Processing, pp. 543-547, Oct. 1997.[11] Y. Deng and B.S. Manjunath, “NeTra-V: Toward an Object-Based Video Representation,” IEEE Trans. Circuits and Systems for Video Technology, vol. 8, no. 5, pp. 616-627, 1998.[12] B. Duc, P. Schroeter, and J. Bigun, Spatio-Temporal Robust Motion Estimation and Segmentation Proc. Sixth Int'l Conf. Computer Analysis Images and Patterns, pp. 238-245, 1995.[13] R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis. John Wiley and Sons, 1973.[14] A. Hampapur, A. Gupta, B. Horowitz, C.-F. Shu, C. Fuller, J. Bach, M. Gorkani, and R. Jain, Virage Video Engine Proc. SPIE Conf., vol. 3022, pp. 188-200, 1997.[15] C. Fowlkes, S. Belongie, and J. Malik, Efficient Spatiotemporal Grouping Using the Nystrom Method Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), vol. I, pp. 231-238, 2001.[16] H. Greenspan, J. Goldberger, and A. Mayer, A Probabilistic Framework for Spatio-Temporal Video Representation and Indexing Proc. IEEE European Conf. Computer Vision (ECCV '02), vol. 4, pp. 461-475, 2002.[17] H. Greenspan, J. Goldberger, and L. Ridel, A Continuous Probabilistic Framework for Image Matching J. Computer Vision and Image Understanding, vol. 84, pp. 384-406, 2001.[18] B. Horn and B. Schunck, Determining Optical Flow Artificial Intelligence, vol. 17, pp. 185-203, 1981.[19] F. Idris and S. Panchanathan, Review of Image and Video Indexing Techniques J. Visual Comm. and Image Representation, vol. 8, no. 2, pp. 146-166, 1997.[20] G. Iyengar and A. B. Lippman, Videobook: An Experiment in Characterization of Video Proc. IEEE Int'l Conf. Image Processing, vol. 3, pp. 855-858, 1996.[21] S. Khan and M. Shah, Object Based Segmentation of Video Using Color, Motion, and Spatial Information Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), vol. II, pp. 746-751, 2001.[22] V. Koble, D. Doermann, and K. Lin, Archiving, Indexing, and Retieval of Video in the Compressed Domain Proc. SPIE Conf., vol. 2916, pp. 78-89, 1996.[23] R. Megret and D. DeMenthon, A Survey of Spatio-Temporal Grouping Techniques Research Report CS-TR-4403, LAMP, Univ. of Maryland, 2002.[24] M.R. Naphade and T.S. Huang, A Probabilistic Framework for Semantic Video Indexing, Filtering, and Retrieval IEEE Trans. Multimedia, vol. 3, no. 1, pp. 141-151, 2001.[25] P. Salembier and F. Marques, Region-Based Representations of Image and Video: Segmentation Tools for Multimedia Services IEEE Trans. Circuits and Systems for Video Technology, vol. 9, no. 8, pp. 1147-1168, 1999.[26] J. Shi and J. Malik, Motion Segmentation and Tracking Using Normalized Cuts Proc. Sixth Int'l Conf. Computer Vision (ICCV '98), pp. 1154-1160, 1998.[27] J.Y. Wang and E.H. Adelson, Spatio-Temporal Segmentation of Video Data Proc. SPIE, vol. 2182, pp. 120-131, 1994.[28] G. Wyszecki and W. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulae. Wiley, 1982.[29] H.J. Zhang, Y. Gong, S.W. Smoliar, and S.Y. Tan, Automatic Parsing of News Video Proc. Int'l Conf. Multimedia Computing and Systems, pp. 45-54, May 1994.[30] H.J. Zhang and S.W. Smoliar, Developing Power Tools for Video and Retrieval Proc. SPIE Conf.: Storage Retrieval Image and Video Databases, vol. II, vol. 2185, pp. 140-149, Feb. 1994.
Index Terms:
Video representation, video segmentation, detection of events in video, Gaussian mixture model.
Citation:
Hayit Greenspan, Jacob Goldberger, Arnaldo Mayer, "Probabilistic Space-Time Video Modeling via Piecewise GMM," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 3, pp. 384-396, Mar. 2004, doi:10.1109/TPAMI.2004.1262334
