The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2013 vol.35)
pp: 513-526
E. Ricci , Dipt. di Ing. Elettron. e dell'Inf., Univ. degli Studi di Perugia, Perugia, Italy
G. Zen , Dipt. di Ing. e Scienza dell'Inf. DISI, Univ. degli Studi di Trento, Trento, Italy
N. Sebe , Dipt. di Ing. e Scienza dell'Inf. DISI, Univ. degli Studi di Trento, Trento, Italy
S. Messelodi , Fondazione Bruno Kessler (FBK) irst, Trento, Italy
ABSTRACT
In the last decades, many efforts have been devoted to develop methods for automatic scene understanding in the context of video surveillance applications. This paper presents a novel nonobject centric approach for complex scene analysis. Similarly to previous methods, we use low-level cues to individuate atomic activities and create clip histograms. Differently from recent works, the task of discovering high-level activity patterns is formulated as a convex prototype learning problem. This problem results in a simple linear program that can be solved efficiently with standard solvers. The main advantage of our approach is that, using as the objective function the Earth Mover's Distance (EMD), the similarity among elementary activities is taken into account in the learning phase. To improve scalability we also consider some variants of EMD adopting L1 as ground distance for 1D and 2D, linear and circular histograms. In these cases, only the similarity between neighboring atomic activities, corresponding to adjacent histogram bins, is taken into account. Therefore, we also propose an automatic strategy for sorting atomic activities. Experimental results on publicly available datasets show that our method compares favorably with state-of-the-art approaches, often outperforming them.
INDEX TERMS
Histograms, Prototypes, Image analysis, Context, Optimization, Clustering algorithms, Optical imaging, parametric linear programming, Video surveillance, complex scene analysis, earth mover's distance
CITATION
E. Ricci, G. Zen, N. Sebe, S. Messelodi, "A Prototype Learning Framework Using EMD: Application to Complex Scenes Analysis", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 3, pp. 513-526, March 2013, doi:10.1109/TPAMI.2012.131
REFERENCES
[1] F. Nater, H. Grabner, and L. Van Gool, "Temporal Relations in Videos for Unsupervised Activity Analysis," Proc. British Machine Vision Conf., 2011.
[2] M. Marszalek, I. Laptev, and C. Schmid, "Actions in Context," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[3] J.C. Niebles, C.-W. Chen, and L. Fei-Fei, "Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification," Proc. 12th European Conf. Computer Vision, 2010.
[4] M.D. Breitenstein, H. Grabner, and L. Van Gool, "Hunting Nessie —Real-Time Abnormality Detection from Webcams," Proc. IEEE Int'l Workshop Visual Surveillance, 2009.
[5] T. Hospedales, J. Li, S. Gong, and T. Xiang, "Identifying Rare and Subtle Behaviours: A Weakly Supervised Joint Topic Model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 12, pp. 2451-2464, Dec. 2011.
[6] T. Hospedales, S. Gong, and T. Xiang, "A Markov Clustering Topic Model for Mining Behaviour in Video," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[7] D. Kuettel, M.D. Breitenstein, L. Van Gool, and V. Ferrari, "What's Going On? Discovering Spatio-Temporal Dependencies in Dynamic Scenes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[8] J. Varadarajan, R. Emonet, and J.-M. Odobez, "Probabilistic Latent Sequential Motifs: Discovering Temporal Activity Patterns in Video Scenes," Proc. British Machine Vision Conf., 2010.
[9] X. Wang, K. Tieu, and W.E.L. Grimson, "Learning Semantic Scene Models by Trajectory Analysis," Proc. European Conf. Computer Vision, 2006.
[10] E.E. Zelniker, S.G. Gong, and T. Xiang, "Global Abnormal Detection Using a Network of CCTV Cameras," Proc. Workshop Visual Surveillance, 2008.
[11] G. Zen, E. Ricci, S. Messelodi, and N. Sebe, "Sorting Atomic Activities for Discovering Spatio-temporal Patterns in Dynamic Scenes," Proc. Int'l Conf. Image Analysis and Processing, 2011.
[12] G. Zen and E. Ricci, "Earth Mover's Prototypes: A Convex Learning Approach for Discovering Activity Patterns in Dynamic Scenes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[13] T. Xiang and S. Gong, "Video Behavior Profiling for Anomaly Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 893-908, May 2008.
[14] X. Wang, X. Ma, and W.E.L. Grimson, "Unsupervised Activity Perception in Crowded and Complicated Scenes Using Hierarchical Bayesian Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 3, pp. 539-555, Mar. 2009.
[15] Y. Yang, J. Liu, and M. Shah, "Video Scene Understanding Using Multi-Scale Analysis," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[16] V. Chandola, A. Banerjee, and V. Kumar, "Anomaly Detection: A Survey," ACM Computing Surveys, vol. 41, no. 3, pp. 1-58, 2009.
[17] M. Breunig, H. Kriegel, R. Ng, and J. Sander, "LOF: Identifying Density-Based Local Outliers," Proc. ACM SIGMOD Int'l Conf. Management of Data, 2000.
[18] J. Tang and Z. Chen, and A.W. Fu, and D.W. Cheung, "Enhancing Effectiveness of Outlier Detections for Low Density Patterns," Proc. Sixth Pacific-Asia Conf. Advances in Knowledge Discovery and Data Mining, 2002.
[19] J. Friedman, T. Hastie, H. Hofling, and R. Tibshirani, "Pathwise Coordinate Optimization," Annals of Applied Statistics, vol. 1, pp. 302-332, 2007.
[20] Y. Yao and Y. Lee, "Another Look at Linear Programming for Feature Selection via Methods of Regularization," Technical Report 800, Dept. of Statistics, Ohio State Univ., 2007.
[21] R. Sandler and M. Lindenbaum, "Nonnegative Matrix Factorization with Earth Mover's Distance Metric," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[22] D. Lashkari and P. Golland, "Convex Clustering with Exemplar-Based Models," Proc. Advances in Neural Information Processing Systems, 2008.
[23] S. Nowozin and S. Jegelka, "Solution Stability in Linear Programming Relaxations: Graph Partitioning and Unsupervised Learning," Proc. Int'l Conf. Machine Learning, 2009.
[24] C. Stauffer and W.E.L. Grimson, "Adaptive Background Mixture Models for Real-Time Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[25] Y. Rubner, C. Tomasi, and L.J. Guibas, "The Earth Mover's Distance as a Metric for Image Retrieval," Int'l J. Computer Vision, vol. 40, no. 2, pp. 99-121, 2000.
[26] H. Ling and K. Okada, "An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 5, pp. 840-853, May 2007.
[27] O. Pele and M. Werman, "A Linear Time Histogram Metric for Improved SIFT Matching," Proc. European Conf. Computer Vision, 2008.
[28] O. Pele and M. Werman, "Fast and Robust Earth Mover's Distances," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[29] K. Murty, Linear Programming. Wiley, 1983.
[30] D. Bertsimas and J.N. Tsitsiklis, Introduction to Linear Optimization. Athena Scientific, 1997.
[31] J. Li, S. Gong, and T. Xiang, "Scene Segmentation for Behaviour Correlation," Proc. European Conf. Computer Vision, 2008.
[32] L.I. Rudin, S. Osher, and E. Fatemi, "Nonlinear Total Variation Based Noise Removal Algorithms," Physica D, vol. 60, pp. 259-268, 1992.
[33] T. Xiang and S. Gong, "Beyond Tracking: Modelling Activity and Understanding Behaviour," Int'l J. Computer Vision, vol. 67, no. 1, pp. 21-51, 2006.
[34] R. Hamid, A. Johnson, S. Batta, A. Bobick, C. Isbell, and G. Coleman, "Detection and Explanation of Anomalous Activities: Representing Activities as Bags of Event n-Grams," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[35] X. Wang, X. Ma, and E. Grimson, "Unsupervised Activity Perception by Hierarchical Bayesian Models," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[36] T. Duong, H. Bui, D. Phung, and S. Venkatesh, "Activity Recognition and Abnormality Detection with the Switching Hidden Semi-Markov Model," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[37] S. Rosset and J. Zhu, "Piecewise Linear Regularized Solution Paths," Annals of Statistics, vol. 35, no. 3, pp. 1012-1030, 2007.
[38] J. Li, S. Gong, and T. Xiang, "Global Behaviour Inference Using Probabilistic Latent Semantic Analysis," Proc. British Machine Vision Conf., 2008.
[39] Y. Takano and Y. Yamamoto, "Metric-Preserving Reduction of Earth Mover's Distance," Asia-Pacific J. Operational Research, vol. 27, 39-54, 2010.
[40] S. Shirdhonkar and D.W. Jacobs, "Approximate Earth Mover's Distance in Linear Time," Proc. IEEE Computer Vision and Pattern Recognition, 2008.
[41] D. Applegate, T. Dasu, S. Krishnan, and S. Urbanek, "Unsupervised Clustering of Multidimensional Distributions Using Earth Mover Distance," Proc. ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, 2011.
[42] J. Wagner and B. Ommer, "Efficiently Clustering Earth Mover's Distance," Proc. Asian Conf. Computer Vision, 2010.
[43] R. Emonet, J. Varadarajan, and J.M. Odobez, "Extracting and Locating Temporal Motifs in Video Scenes Using a Hierarchical Non Parametric Bayesian Model," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
33 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool