The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2013 vol.35)
pp: 1606-1621
A. Mumtaz , Dept. of Comput. Sci., City Univ. of Hong Kong, Kowloon, China
E. Coviello , Dept. of Electr. & Comput. Eng., Univ. of California, San Diego, La Jolla, CA, USA
G. R. G. Lanckriet , Dept. of Electr. & Comput. Eng., Univ. of California, San Diego, La Jolla, CA, USA
A. B. Chan , Dept. of Comput. Sci., City Univ. of Hong Kong, Kowloon, China
ABSTRACT
Dynamic texture (DT) is a probabilistic generative model, defined over space and time, that represents a video as the output of a linear dynamical system (LDS). The DT model has been applied to a wide variety of computer vision problems, such as motion segmentation, motion classification, and video registration. In this paper, we derive a new algorithm for clustering DT models that is based on the hierarchical EM algorithm. The proposed clustering algorithm is capable of both clustering DTs and learning novel DT cluster centers that are representative of the cluster members in a manner that is consistent with the underlying generative probabilistic model of the DT. We also derive an efficient recursive algorithm for sensitivity analysis of the discrete-time Kalman smoothing filter, which is used as the basis for computing expectations in the E-step of the HEM algorithm. Finally, we demonstrate the efficacy of the clustering algorithm on several applications in motion analysis, including hierarchical motion clustering, semantic motion annotation, and learning bag-of-systems (BoS) codebooks for dynamic texture recognition.
INDEX TERMS
Heuristic algorithms, Clustering algorithms, Computational modeling, Algorithm design and analysis, Dynamics, Kalman filters, Nickel, sensitivity analysis, Dynamic textures, expectation maximization, Kalman filter, bag of systems, video annotation
CITATION
A. Mumtaz, E. Coviello, G. R. G. Lanckriet, A. B. Chan, "Clustering Dynamic Textures with the Hierarchical EM Algorithm for Modeling Video", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 7, pp. 1606-1621, July 2013, doi:10.1109/TPAMI.2012.236
REFERENCES
[1] B. Horn and B. Schunk, "Determining Optical Flow," Artificial Intelligence, vol. 17, pp. 185-204, 1981.
[2] B. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proc. DARPA Image Understanding Workshop, pp. 121-130, 1981.
[3] G. Doretto, A. Chiuso, Y.N. Wu, and S. Soatto, "Dynamic Textures," Int'l J. Computer Vision, vol. 51, no. 2, pp. 91-109, 2003.
[4] A.W. Fitzgibbon, "Stochastic Rigidity: Image Registration for Nowhere-Static Scenes," Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp. 662-670, 2001.
[5] A. Ravichandran and R. Vidal, "Video Registration Using Dynamic Textures," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 1 pp. 158-171, Jan. 2011.
[6] G. Doretto, D. Cremers, P. Favaro, and S. Soatto, "Dynamic Texture Segmentation," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1236-1242, 2003.
[7] A. Ghoreyshi and R. Vidal, "Segmenting Dynamic Textures with Ising Descriptors, ARX Models and Level Sets," Proc. Dynamical Vision Workshop in the European Conf. Computer Vision, 2006.
[8] A.B. Chan and N. Vasconcelos, "Modeling, Clustering, and Segmenting Video with Mixtures of Dynamic Textures," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 909-926, May 2008.
[9] R. Vidal and A. Ravichandran, "Optical Flow Estimation and Segmentation of Multiple Moving Dynamic Textures," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 516-521, 2005.
[10] A.B. Chan and N. Vasconcelos, "Layered Dynamic Textures," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 10, pp. 1862-1879, Oct. 2009.
[11] R. Chaudry, A. Ravichandran, G. Hager, and R. Vidal, "Histograms of Oriented Optical Flow and Binet-Cauchy Kernels on Nonlinear Dynamical Systems for the Recognition of Human Actions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[12] A. Bissacco, A. Chiuso, and S. Soatto, "Classification and Recognition of Dynamical Models: The Role of Phase, Independent Components, Kernels and Optimal Transport," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1958-1972, Nov. 2007.
[13] P. Saisan, G. Doretto, Y. Wu, and S. Soatto, "Dynamic Texture Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 58-63, 2001.
[14] A.B. Chan and N. Vasconcelos, "Probabilistic Kernels for the Classification of Auto-Regressive Visual Processes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 846-851, 2005.
[15] S.V.N. Vishwanathan, A.J. Smola, and R. Vidal, "Binet-Cauchy Kernels on Dynamical Systems and Its Application to the Analysis of Dynamic Scenes," Int'l J. Computer Vision, vol. 73, no. 1, pp. 95-119, 2007.
[16] R. Vidal and P. Favaro, "Dynamicboost: Boosting Time Series Generated by Dynamical Systems," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[17] A.B. Chan and N. Vasconcelos, "Classifying Video with Kernel Dynamic Textures," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[18] A. Ravichandran, R. Chaudhry, and R. Vidal, "View-Invariant Dynamic Texture Recognition Using a Bag of Dynamical Systems," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[19] B. Ghanem and N. Ahuja, "Phase Based Modelling of Dynamic Textures," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[20] F. Woolfe and A. Fitzgibbon, "Shift-Invariant Dynamic Texture Recognition," Proc. Ninth European Conf. Computer Vision, 2006.
[21] H. Cetingul and R. Vidal, "Intrinsic Mean Shift for Clustering on Stiefel and Grassmann Manifolds," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[22] A. Goh and R. Vidal, "Clustering and Dimensionality Reduction on Riemannian Manifolds," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[23] N. Vasconcelos and A. Lippman, "Learning Mixture Hierarchies," Proc. Neural Information Processing Systems Conf., 1998.
[24] A.B. Chan, E. Coviello, and G. Lanckriet, "Clustering Dynamic Textures with the Hierarchical EM Algorithm," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[25] G. Carneiro, A.B. Chan, P.J. Moreno, and N. Vasconcelos, "Supervised Learning of Semantic Classes for Image Annotation and Retrieval," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 3, pp. 394-410, Mar. 2007.
[26] A. Gelb, Applied Optimal Estimation. MIT Press, 1974.
[27] N. Vasconcelos, "Image Indexing with Mixture Hierarchies," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001.
[28] A. Banerjee, S. Merugu, I. Dhillon, and J. Ghosh, "Clustering with Bregman Divergences," J. Machine Learning Research, vol. 6, pp. 1705-1749, 2005.
[29] J.V. Davis and I. Dhillon, "Differential Entropic Clustering of Multivariate Gaussians," Proc. Advances in Neural Information Processing Systems Conf., 2006.
[30] J. Goldberger and S. Roweis, "Hierarchical Clustering of a Mixture Model," Proc. Advances in Neural Information Processing Systems Conf., pp. 505-512, 2005.
[31] R.E. Griffin and A.P. Sage, "Sensitivity Analysis of Discrete Filtering and Smoothing Algorithms," AIAA J., vol. 7, pp. 1890-1897, Oct. 1969.
[32] J. Wall, A. Willsky, and N. Sandell, "On the Fixed-Interval Smoothing Problem." Stochastics., vol. 5, pp. 1-41, 1981.
[33] E. Coviello, A. Chan, and G. Lanckriet, "Time Series Models for Semantic Music Annotation," IEEE Trans. Audio, Speech, and Language Processing, vol. 19, no. 5, pp. 1343-1359, July 2011.
[34] R.H. Shumway and D.S. Stoffer, "An Approach to Time Series Smoothing and Forecasting Using the EM Algorithm," J. Time Series Analysis, vol. 3, no. 4, pp. 253-264, 1982.
[35] A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Soc. B, vol. 39, pp. 1-38, 1977.
[36] S.M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory. Prentice-Hall, 1993.
[37] R. Péteri, S. Fazekas, and M.J. Huiskes, "DynTex: A Comprehensive Database of Dynamic Textures," Pattern Recognition Letters, vol. 31, no. 12, pp. 1627-1632, 2010.
[38] A. Ravichandran, R. Chaudhry, and R. Vidal, "Categorizing Dynamic Textures Using a Bag of Dynamical Systems," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 2, pp. 342-353, Feb. 2013.
[39] K.G. Derpanis and R.P. Wildes, "Dynamic Texture Recognition Based on Distributions of Spacetime Oriented Structure," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[40] G. Zhao and M. Pietikainen, "Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 915-928, June 2007.
[41] C. Chang and C. Lin, "Libsvm: A Library for Support Vector Machines," ACM Trans. Intelligent Systems and Technology, vol. 2, 2011.
[42] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction, second ed. Springer, http://www-stat.stanford.edu/tibsElemStatLearn /, 2009.
[43] D.M. Blei and M.I. Jordan, "Variational Inference for Dirichlet Process Mixtures," Bayesian Analysis, vol. 1, pp. 121-144, 2005.
28 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool