Issue No. 07 - July (2014 vol. 36)
Dian Gong , , Institute for Robotics and Intelligence Systems, University of Southern California, Los Angeles, CA, USA
Gerard Medioni , , Institute for Robotics and Intelligence Systems, University of Southern California, Los Angeles, CA, USA
Xuemei Zhao , , Institute for Robotics and Intelligence Systems, University of Southern California, Los Angeles, CA, USA
We address the problem of structure learning of human motion in order to recognize actions from a continuous monocular motion sequence of an arbitrary person from an arbitrary viewpoint. Human motion sequences are represented by multivariate time series in the joint-trajectories space. Under this structured time series framework, we first propose Kernelized Temporal Cut (KTC), an extension of previous works on change-point detection by incorporating Hilbert space embedding of distributions, to handle the nonparametric and high dimensionality issues of human motions. Experimental results demonstrate the effectiveness of our approach, which yields realtime segmentation, and produces high action segmentation accuracy. Second, a spatio-temporal manifold framework is proposed to model the latent structure of time series data. Then an efficient spatio-temporal alignment algorithm Dynamic Manifold Warping (DMW) is proposed for multivariate time series to calculate motion similarity between action sequences (segments). Furthermore, by combining the temporal segmentation algorithm and the alignment algorithm, online human action recognition can be performed by associating a few labeled examples from motion capture data. The results on human motion capture data and 3D depth sensor data demonstrate the effectiveness of the proposed approach in automatically segmenting and recognizing motion sequences, and its ability to handle noisy and partially occluded data, in the transfer learning module.
Motion segmentation, Three-dimensional displays, Kernel, Time series analysis, Manifolds, Heuristic algorithms, Hidden Markov models,Computer vision, Machine learning
Dian Gong, Gerard Medioni, Xuemei Zhao, "Structured Time Series Analysis for Human Action Segmentation and Recognition", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 36, no. , pp. 1414-1427, July 2014, doi:10.1109/TPAMI.2013.244