This Article 
 Bibliographic References 
 Add to: 
Classification and Recognition of Dynamical Models: The Role of Phase, Independent Components, Kernels and Optimal Transport
November 2007 (vol. 29 no. 11)
pp. 1958-1972
We address the problem of performing decision tasks, and in particular classification and recognition, in the space of dynamical models in order to compare time series of data. Motivated by the application of recognition of human motion in image sequences, we consider a class of models that include linear dynamics, both stable and marginally stable (periodic), both minimum and non-minimum phase, driven by non-Gaussian processes. This requires extending existing learning and system identification algorithms to handle periodic modes and nonminimum phase behavior, while taking into account higher-order statistics of the data. Once a model is identified, we define a kernel-based cord distance between models that includes their dynamics, their initial conditions as well as input distribution. This is made possible by a novel kernel defined between two arbitrary (non-Gaussian) distributions, which is computed by efficiently solving an optimal transport problem. We validate our choice of models, inference algorithm, and distance on the tasks of human motion synthesis (sample paths of the learned models), and recognition (nearest-neighbor classification in the computed distance). However, our work can be applied more broadly where one needs to compare historical data while taking into account periodic trends, non-minimum phase behavior, and non-Gaussian input distributions.

[1] T. Kailath, A.H. Sayed, and B. Hassibi, Linear Estimation. Prentice-Hall, 2000.
[2] J. Levine, “Finite Dimensional Filters for a Class of Nonlinear Systems and Immersion in a Linear System,” SIAM J. Control and Optimization, vol. 25, no. 6, pp. 1430-1439, 1987.
[3] M.J. Hinich, “Testing for Gaussianity and Linearity of a Stationary Time Series,” J. Time Series Analysis, vol. 3, pp. 169-176, 1982.
[4] I. Goethals, K. Pelckmans, J.A.K. Suykens, and B.D. Moor, “Subspace Identification of Hammerstein Systems Using Least Squares Support Vector Machines,” IEEE Trans. Automatic Control, vol. 50, no. 10, pp. 1509-1519, Oct. 2005.
[5] L. Ljung, System Identification Theory for the User. Prentice-Hall, 1997.
[6] T. Söderström and P. Stoica, System Identification. Prentice-Hall, 1989.
[7] A. Lindquist and G. Picci, “A Geometric Approach to Modeling and Estimation of Linear Stochastic Systems,” J. Math. Systems, Estimation, and Control, vol. 1, pp. 241-333, 1991.
[8] P.V. Overschee and B.D. Moor, Subspace Identification for Linear Systems: Theory, Implementation, Applications. Kluwer Academic, 1996.
[9] A. Cichocki and S. Amari, Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. John Wiley & Sons, 2003.
[10] A. Swami, G. Giannakis, and S. Shamsunder, “Multichannel ARMA Processes,” IEEE Trans. Signal Processing, vol. 42, no. 4, pp.898-913, 1994.
[11] K.D. Coch and B.D. Moor, “Subspace Angles and Distances between Arma Models,” Proc. Int'l Symp. Math. Theory of Networks and Systems, 2000.
[12] R. Martin, “A Metric for Arma Processes,” IEEE Trans. Signal Processing, vol. 48, no. 4, pp. 1164-1170, 2000.
[13] M.C. Mazzaro, M. Sznaier, and O.I. Camps, “A Model (in)Validation Approach to Gait Classification,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1820-1825, Nov. 2005.
[14] S. Vishwanathan, R. Vidal, and A.J. Smola, “Binet-Cauchy Kernels on Dynamical Systems and Its Application to the Analysis of Dynamic Scenes,” Int'l J. Computer Vision, 2005.
[15] Y. Rozanov, Stationary Random Processes. Holden-Day, 1967.
[16] A. Papoulis, “Predictable Processes and Wold's Decomposition: A Review,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 33, no. 4, pp. 933-938, 1985.
[17] R. Brockett, Finite Dimensional Linear Systems. John Wiley & Sons, 1970.
[18] P. Mullhaupt, B. Srinivasan, and D. Bonvin, “On the Nonminimum-Phase Characteristics of Two-Link Underactuated Mechanical Systems,” Proc. 37th IEEE Conf. Decision and Control, 1998.
[19] P. Saisan, A. Bissacco, A. Chiuso, and S. Soatto, “Modeling and Synthesis of Facial Motion Driven by Speech,” Proc. Eighth European Conf. Computer Vision, 2004.
[20] P. Comon, “Independent Component Analysis: A New Concept?” Signal Processing, vol. 36, pp. 287-314, 1994.
[21] T. Blaschke and L. Wiskott, “Cubica: Independent Component Analysis by Simultaneous Third- and Fourth-Order Cumulant Diagonalization,” IEEE Trans. Signal Processing, vol. 52, pp. 1250-1256, 2004.
[22] D. Bauer and M. Wagner, “Estimating Cointegrated Systems Using Subspace Algorithms,” J. Econometrics, vol. 111, pp. 47-84, 2002.
[23] A. Lindquist and G. Picci, “Geometric Methods for State-Space Identification,” Identification, Adaptation, Learning, S. Bittanti and G. Picci, eds., Springer, pp. 1-69, 1996.
[24] R.O. Schmidt, “A Signal Subspace Approach to Multiple Emitter Location and Spectral Estimation,” PhD dissertation, Stanford Univ., Nov. 1981.
[25] R.H. Roy, “Esprit—A Subspace Rotation Approach to Estimation of Parameters of Cisoids in Noise,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 34, pp. 1340-1342, Oct. 1986.
[26] M. Kristensson, M. Jansson, and B. Ottersten, “Further Results and Insights on Subspace-Based Sinusoidal Frequency Estimation,” IEEE Trans. Signal Processing, vol. 49, no. 12, pp. 2962-2974, 2001.
[27] A. Eriksson, P. Stoica, and T. Soderstrom, “Markov-Based Eigenanalysis Method for Frequency Estimation,” IEEE Trans. Signal Processing, vol. 42, no. 3, pp. 586-594, 1994.
[28] P. Van Overschee and B. De Moor, “Subspace Algorithms for the Stochastic Identification Problem,” Automatica, vol. 29, pp. 649-660, 1993.
[29] A. Chiuso and G. Picci, “On the Ill-Conditioning of Subspace Identification with Inputs,” Automatica, vol. 40, no. 4, pp. 575-589, 2004.
[30] D. Bauer, “Asymptotic Properties of Subspace Estimators,” Automatica, vol. 41, pp. 359-376, 2005.
[31] A. Chiuso, “Asymptotic Variance of Closed-Loop Subspace Identification Algorithms,” IEEE Trans. Automatic Control, vol. 51, no. 8, pp. 1299-1314, 2006.
[32] G. Golub and C. Van Loan, Matrix Computation, second ed. Johns Hopkins Univ. Press, 1989.
[33] R. Horn and C. Johnson, Matrix Analysis. Cambridge Univ. Press, 1985.
[34] G. Picci and T. Katayama, “Stochastic Realization with Exogenous Inputs and “Subspace Methods” Identification,” Signal Processing, vol. 52, pp. 145-160, 1996.
[35] A. Chiuso and G. Picci, “Subspace Identification by Orthogonal Decomposition,” Proc. 14th Int'l Federation of Automatic Control (IFAC) World Congress, vol. I, pp. 241-246, 1999.
[36] A. Bissacco, A. Chiuso, and S. Soatto, “Classification and Recognition of Dynamical Models,” technical report, Computer Science Dept., Univ. of California, Los Angeles, 2006.
[37] M. Boumahdi, “Blind Identification Using the Kurtosis with Applications to Field Data,” Signal Processing, vol. 48, pp. 205-216, 1996.
[38] J. Mourjopoulos, P.M. Clarkson, and J.K. Hammond, “A Comparative Study of Least-Squares and Homomorphic Techniques for the Inversion of Mixed Phase Signal,” Proc. IEEE Int'l Conf. Acoustics, Speech and Signal Processing, pp. 1858-1861, 1982.
[39] C.L. Mallows, “A Note on Asymptotic Joint Normality,” Annals of Math. Statistics, vol. 43, pp. 508-515, 1972.
[40] P.J. Bickel and D.A. Freedman, “Some Asymptotic Theory for the Bootstrap,” Annals of Statistics, vol. 9, no. 6, pp. 1196-1217, 1981.
[41] B. Schoelkopf and A.J. Smola, Learning with Kernels. Massachusetts Inst. of Technology (MIT) Press, 2002.
[42] H.W. Kuhn, “The Hungarian Method for the Assignment Problem,” Naval Research Logistics Quarterly, vol. 2, no. 83, 1955.
[43] “Carnegie-Mellon Mocap Database,” http:/, 2003.
[44] R. Gross and J. Shi, “The CMU Motion of Body (Mobo) Database,” technical report, Robotics Inst., Carnegie Mellon Univ., 2001.
[45] G. Doretto, A. Chiuso, Y.N. Wu, and S. Soatto, “Dynamic Textures,” Int'l J. Computer Vision, vol. 51, no. 2, pp. 91-109, 2003.
[46] F. Cuzzolin, “Using Bilinear Models for View-Invariant Action and Identity Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.

Index Terms:
System Identification, Blind Deconvolution, Non-minimum Phase, Distance, Kernel, Hammerstein models, Optimal Transport, Wasserstein models, Non-Gaussian models, Learning, Time Series, Higher-Order Statistics
Alessandro Bissacco, Alessandro Chiuso, Stefano Soatto, "Classification and Recognition of Dynamical Models: The Role of Phase, Independent Components, Kernels and Optimal Transport," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1958-1972, Nov. 2007, doi:10.1109/TPAMI.2007.1101
Usage of this product signifies your acceptance of the Terms of Use.