The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.08 - August (2009 vol.31)
pp: 1472-1485
Imran Saleemi , University of Central Florida, Orlando
Khurram Shafique , Object Video Inc., Reston
Mubarak Shah , University of Central Florida, Orlando
ABSTRACT
We propose a novel method to model and learn the scene activity, observed by a static camera. The proposed model is very general and can be applied for solution of a variety of problems. The motion patterns of objects in the scene are modeled in the form of a multivariate nonparametric probability density function of spatiotemporal variables (object locations and transition times between them). Kernel Density Estimation is used to learn this model in a completely unsupervised fashion. Learning is accomplished by observing the trajectories of objects by a static camera over extended periods of time. It encodes the probabilistic nature of the behavior of moving objects in the scene and is useful for activity analysis applications, such as persistent tracking and anomalous motion detection. In addition, the model also captures salient scene features, such as the areas of occlusion and most likely paths. Once the model is learned, we use a unified Markov Chain Monte Carlo (MCMC)-based framework for generating the most likely paths in the scene, improving foreground detection, persistent labeling of objects during tracking, and deciding whether a given trajectory represents an anomaly to the observed motion patterns. Experiments with real-world videos are reported which validate the proposed approach.
INDEX TERMS
Vision and scene understanding, Markov processes, machine learning, tracking, kernel density estimation, Metropolis-Hastings, Markov Chain Monte Carlo.
CITATION
Imran Saleemi, Khurram Shafique, Mubarak Shah, "Probabilistic Modeling of Scene Dynamics for Applications in Visual Surveillance", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 8, pp. 1472-1485, August 2009, doi:10.1109/TPAMI.2008.175
REFERENCES
[1] “Special Issue on Video Communications, Processing, and Understanding for Third Generation Surveillance Systems,” Proc. IEEE, vol. 89, no. 10, Oct. 2001.
[2] R. Collins, J. Lipton, and T. Kanade, “Introduction to the Special Section on Video Surveillance,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, Aug. 2000.
[3] O. Javed, K. Shafique, and M. Shah, “Automated Visual Surveillance in Realistic Scenarios,” IEEE Multimedia, pp. 30-39, Jan.-Mar. 2007.
[4] I. Biederman, “On the Semantics of a Glance at a Scene,” Perceptual Organization, pp. 213-253, Lawrence Erlbaum Assoc., 1981.
[5] A. Torralba, “Contextual Influences on Saliency,” Neurobiology of Attention, 2005.
[6] K. Tieu, G. Dalley, and W.E.L. Grimson, “Inference of Non-Overlapping Camera Network Topology by Measuring Statistical Dependence,” Proc. 10th IEEE Int'l Conf. Computer Vision, pp. 1842-1849, 2005.
[7] T.J. Ellis, D. Makris, and J.K. Black, “Learning a Multi-Camera Topology,” Proc. Joint IEEE Int'l Workshop Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2003.
[8] C. Stauffer, “Learning to Track Objects through Unobserved Regions,” Proc. IEEE Workshop Motion and Video Computing, vol. 2, pp. 96-102, 2005.
[9] O. Javed, K. Shafique, Z. Rasheed, and M. Shah, “Modeling Inter-Camera Space-Time and Appearance Relationships for Tracking across Non-Overlapping Views,” Computer Vision and Image Understanding, vol. 109, no. 2, Feb. 2008.
[10] H. Buxton, “Generative Models for Learning and Understanding Dynamic Scene Activity,” Proc. Generative Model Based Vision Workshop, 2002.
[11] A. Hunter, J. Owens, and M. Carpenter, “A Neural System for Automated CCTV Surveillance,” IEE Intelligent Distributed Surveillance Systems, 2003.
[12] J. Owens and A. Hunter, “Application of the Self-Organising Map to Trajectory Classification,” Proc. Third IEEE Int'l Workshop Visual Surveillance, 2000.
[13] N. Johnson and D.C. Hogg, “Learning the Distribution of Object Trajectories for Event Recognition,” Image and Vision Computing, vol. 14, no. 8, pp. 609-615, Aug. 1996.
[14] J.H. Fernyhough, A.G. Cohn, and D.C. Hogg, “Generation of Semantic Regions from Image Sequences,” Proc. Fourth European Conf. Computer Vision, 1996.
[15] R.J. Howard and H. Buxton, “Analogical Representation of Spatial Events, for Understanding Traffic Behaviour,” Proc. 10th European Conf. Artificial Intelligence, 1992.
[16] E.B. Koller-Meier and L. Van Gool, “Modeling and Recognition of Human Actions Using a Stochastic Approached,” Proc. Second European Workshop Advanced Video-Based Surveillance Systems, 2001.
[17] J. Lou, Q. Liu, T. Tan, and W. Hu, “Semantic Interpretation of Object Activities in a Surveillance System,” Proc. 16th Int'l Conf. Pattern Recognition, 2002.
[18] W.E.L. Grimson, C. Stauffer, R. Romano, and L. Lee, “Using Adaptive Tracking to Classify and Monitor Activities in a Site,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, 1998.
[19] C. Stauffer and W.E.L. Grimson, “Learning Patterns of Activity Using Real Time Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 747-767, Aug. 2000.
[20] C. Stauffer, “Estimating Tracking Sources and Sinks,” Proc. Second IEEE Event Mining Workshop, 2003.
[21] P. Remagnino and G.A. Jones, “Classifying Surveillance Events from Attributes and Behaviour,” Proc. British Machine Vision Conf., 2001.
[22] M. Walter, A. Psarrou, and S. Gong, “Learning Prior and Observation Augmented Density Models for Behaviour Recognition,” Proc. British Machine Vision Conf., 1999.
[23] A. Galata, N. Johnson, and D. Hogg, “Learning Variable Length Markov Models of Behaviour,” Computer Vision and Image Understanding, vol. 81, no. 3, pp. 398-413, 2001.
[24] T. Huang and S. Russell, “Object Identification in a Bayesian Context,” Proc. 15th Int'l Joint Conf. Artificial Intelligence, 1997.
[25] V. Kettnaker and R. Zabih, “Bayesian Multi-Camera Surveillance,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, 1999.
[26] S.L. Dockstader and A.M. Tekalp, “Multiple Camera Fusion for Multi-Object Tracking,” Proc. IEEE Workshop Multi-Object Tracking, 2001.
[27] D. Hoiem, A. Efros, and M. Hebert, “Putting Objects in Perspective,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, 2006.
[28] R. Rosales and S. Sclaroff, “Improved Tracking of Multiple Humans with Trajectory Prediction and Occlusion Modeling,” Proc. CVPR Workshop Interpretation of Visual Motion, 1998.
[29] R. Kaucic, A. Perera, G. Brooksby, J. Kaufhold, and A. Hoogs, “A Unified Framework for Tracking through Occlusions and across Sensor Gaps,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[30] X. Wang, K. Tieu, and E. Grimson, “Learning Semantic Scene Models by Trajectory Analysis,” Proc. Ninth European Conf. Computer Vision, 2006.
[31] A. Perera, C. Srinivas, A. Hoogs, G. Brooksby, and W. Hu, “Multi-Object Tracking through Simultaneous Long Occlusions and Split-Merge Conditions,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 666-673, 2006.
[32] W. Hu, X. Xiao, Z. Fu, D. Xie, T. Tan, and S. Maybank, “A System for Learning Statistical Motion Patterns,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1450-1464, Sept. 2006.
[33] E. Parzen, “On the Estimation of a Probability Density Function and Mode,” Annals of Math. Statistics, vol. 33, pp. 1065-1076, 1962.
[34] R. Duda, P. Hart, and D. Stork, Pattern Classification, second ed. Wiley Interscience, 2001.
[35] B. Turlach, “Bandwidth Selection in Kernel Density Estimation: A Review,” Institut de Statistique, 1993.
[36] B.W. Silverman, Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1986.
[37] J.A. Benediktsson and P.H. Swain, “Consensus Theoretic Classification Methods,” IEEE Trans. Systems, Man, and Cybernetics, vol. 22, pp. 688-704, 1992.
[38] G. Hinton, “Products of Experts,” Proc. Ninth Int'l Conf. Artificial Neural Networks, pp. 1-6, 1999.
[39] K. Shafique and M. Shah, “A Non-Iterative Greedy Algorithm for Multi-Frame Point Correspondence,” Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
[40] L. Greengard and J. Strain, “The Fast Gauss Transform,” SIAM J. Scientific and Statistical Computing, vol. 12, no. 1, pp. 79-94, 1991.
[41] A. Elgammal, R. Duraiswami, and L. Davis, “The Fast Gauss Transform for Efficient Kernel Density Evaluation with Applications in Computer Vision,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 11, pp. 1499-1504, Nov. 2003.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool