This Article 
 Bibliographic References 
 Add to: 
Matching Shape Sequences in Video with Applications in Human Movement Analysis
December 2005 (vol. 27 no. 12)
pp. 1896-1909
We present an approach for comparing two sequences of deforming shapes using both parametric models and nonparametric methods. In our approach, Kendall's definition of shape is used for feature extraction. Since the shape feature rests on a non-Euclidean manifold, we propose parametric models like the autoregressive model and autoregressive moving average model on the tangent space and demonstrate the ability of these models to capture the nature of shape deformations using experiments on gait-based human recognition. The nonparametric model is based on Dynamic Time-Warping. We suggest a modification of the Dynamic time-warping algorithm to include the nature of the non-Euclidean space in which the shape deformations take place. We also show the efficacy of this algorithm by its application to gait-based human recognition. We exploit the shape deformations of a person's silhouette as a discriminating feature and provide recognition results using the nonparametric model. Our analysis leads to some interesting observations on the role of shape and kinematics in automated gait-based person authentication.

[1] S. Loncaric, “A Survey of Shape Analysis Techniques,” Pattern Recognition, vol. 31, no. 8, pp. 983-1001, 1998.
[2] R.C. Veltkamp and M. Hagedoorn, “State of the Art in Shape Matching,” Technical Report UU-CS-1999-27, Utrecht, vol. 27, 1999.
[3] T. Pavlidis, “A Review of Algorithms for Shape Analysis,” Computer Graphics and Image Processing, vol. 7, pp. 243-258, 1978.
[4] R. Kashyap and R. Chellappa, “Stochastic Models for Closed Boundary Analysis: Representation and Reconstruction,” IEEE Trans. Information Theory, vol. 27, pp. 627-637, 1981.
[5] E. Persoon and K. Fu, “Shape Discrimination Using Fourier Descriptors,” IEEE Trans. Systems, Man, and Cybernetics, vol. 7, no. 3, pp. 170-179, Mar. 1977.
[6] S. Belongie, J. Malik, and J. Puzicha, “Shape Matching and Object Recognition Using Shape Contexts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, pp. 509-522, Apr. 2002.
[7] H. Blum and R. Nagel, “Shape Description Using Weighted Symmetric Axis Features,” Pattern Recognition, vol. 10, pp. 167-180, 1978.
[8] M. Hu, “Visual Pattern Recognition by Moment Invariants,” IRE Trans. Information Theory, vol. 8, pp. 179-187, 1962.
[9] A. Khotanzad and Y. Hong, “Invariant Image Recognition by Zernike Moments,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 5, pp. 489-497, 1990.
[10] C.C. Chen, “Improved Moment Invariants for Shape Discrimination,” Pattern Recognition, vol. 26, no. 5, pp. 683-686, 1993.
[11] A. Goshtasby, “Description and Discrimination of Planar Shapes Using Shape Matrices,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, pp. 738-743, 1985.
[12] S. Parui, E. Sarma, and D. Majumder, “How to Discriminate Shapes Using the Shape Vector,” Pattern Recognition Letters, vol. 4, pp. 201-204, 1986.
[13] H. Freeman, “On the Encoding of Arbitrary Geometric Configurations,” IRE Trans. Electronic Computers, vol. 10, pp. 260-268, 1961.
[14] E. Arkin, L. Chew, D. Huttenlocher, K. Kedem, and J. Mitchell, “An Efficiently Computable Metric for Polygonal Shapes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, pp. 209-216, 1986.
[15] A. Srivastava, W. Mio, E. Klassen, and S. Joshi, “Geometric Analysis of Continuous, Planar Shapes,” Proc. Fourth Int'l Workshop Energy Minimization Methods in Computer Vision and Pattern Recognition, 2003.
[16] D. Kendall, “Shape Manifolds, Procrustean Metrics and Complex Projective Spaces,” Bull. London Math. Soc., vol. 16, pp. 81-121, 1984.
[17] F. Bookstein, “Size and Shape Spaces for Landmark Data in Two Dimensions,” Statistical Science, vol. 1, pp. 181-242, 1986.
[18] I. Dryden and K. Mardia, Statistical Shape Analysis. John Wiley and Sons, 1998.
[19] M. Prentice and K. Mardia, “Shape Changes in the Plane for Landmark Data,” The Annals of Statistics, vol. 23, no. 6, pp. 1960-1974, 1995.
[20] D. Geiger, T. Liu, and R. Kohn, “Representation and Self-Similarity of Shapes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 1, pp. 86-99, Jan. 2003.
[21] R. Berthilsson, “A Statistical Theory of Shape,” Statistical Pattern Recognition, pp. 677-686, 1998.
[22] I. Dryden, “Statistical Shape Analysis in High Level Vision,” Proc. IMA Workshop: Image Analysis and High Level Vision, 2000.
[23] A. Rangarajan, H. Chui, and F. Bookstein, “The Softassign Procrustes Matching Algorithm,” Information Processing in Medical Imaging, pp. 29-42. Springer, 1997.
[24] A. Yezzi and S. Soatto, “Deformotion: Deforming Motion, Shape Average and the Joint Registration and Approximation of Structure in Images,” Int'l J. Computer Vision, vol. 53, no. 2, pp. 153-167, 2003.
[25] P. Maurel and G. Sapiro, “Dynamic Shapes Average,” Proc. Second IEEE Workshop Variational, Geometric and Level Set Methods in Computer Vision, 2003.
[26] N. Vaswani, A. Roy-Chowdhury, and R. Chellappa, “`Shape Activities': A Continuous State HMM for Moving/Deforming Shapes with Application to Abnormal Activity Detection,” IEEE Trans. Image Processing, 2004.
[27] C. Liu and N. Ahuja, “A Model for Dynamic Shape and Its Applications,” Proc. Conf. Computer Vision and Pattern Recognition, 2004.
[28] A. Srivasatava and E. Klassen, “Bayesian Geometric Subspace Tracking,” Advances in Applied Probability, vol. 36, no. 1, pp. 43-56, Mar. 2004.
[29] M. Black and A. Jepson, “Eigentracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation,” Int'l J. Computer Vision, vol. 26, no. 1, pp. 63-84, 1998.
[30] S.D. Mowbray and M.S. Nixon, “Extraction and Recognition of Periodically Deforming Objects by Continuous, Spatio-Temporal Shape Description,” Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 895-901, 2004.
[31] S. Niyogi and E. Adelson, “Analyzing and Recognizing Walking Figures in XYT,” Technical Report 223, MIT Media Lab Vision and Modeling Group, 1994.
[32] J. Phillips, S. Sarkar, I. Robledo, P. Grother, and K. Bowyer, “The Gait Identification Challenge Problem: Data Sets and Baseline Algorithm,” Proc. Int'l Conf. Pattern Recognition, Aug. 2002.
[33] J. Han and B. Bhanu, “Individual Recognition Using Gait Energy Image,” Proc. Workshop Multimodal User Authentication (MMUA 2003), pp. 181-188, Dec. 2003.
[34] L. Wang, H. Ning, W. Hu, and T. Tan, “Gait Recognition Based on Procrustes Shape Analysis,” Proc. Int'l Conf. Image Processing, 2002.
[35] J. Foster, M. Nixon, and A. Prugel-Bennett, “Automatic Gait Recognition Using Area-Based Metrics,” Pattern Recognition Letters, vol. 24, pp. 2489-2497, 2003.
[36] A. Bobick and A. Johnson, “Gait Recognition Using Static Activity-Specific Parameters,” Proc. Conf. Computer Vision and Pattern Recognition, Dec. 2001.
[37] R. Collins, R. Gross, and J. Shi, “Silhoutte Based Human Identification Using Body Shape and Gait,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 351-356, 2002.
[38] A. Kale, A. Rajagopalan, A. Sundaresan, N. Cuntoor, A. Roy-Cowdhury, V. Krueger, and R. Chellappa, “Identification of Humans Using Gait,” IEEE Trans. Image Processing, Sept. 2004.
[39] L. Lee, G. Dalley, and K. Tieu, “Learning Pedestrian Models for Silhoutte Refinement,” Proc. Int'l Conf. Computer Vision, 2003.
[40] D. Tolliver and R. Collins, “Gait Shape Estimation for Identification,” Proc. Fourth Int'l Conf. Audio and Video-Based Biometric Person Authentication, June 2003.
[41] D. Cunado, M. Nash, S. Nixon, and N. Carter, “Gait Extraction and Description by Evidence Gathering,” Proc. Int'l Conf. Audio and Video-Based Biometric Person Authentication, pp. 43-48, 1994.
[42] A. Bissacco, P. Saisan, and S. Soatto, “Gait Recognition Using Dynamic Affine Invariants,” Proc. Int'l Symp. Math. Theory of Networks and Systems, July 2004.
[43] A. Bissacco, A. Chiuso, Y. Ma, and S. Soatto, “Recognition of Human Gaits,” Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 52-57, 2001.
[44] C. Mazzaro, M. Sznaier, O. Camps, S. Soatto, and A. Bissacco, “A Model (In)Validation Approach to Gait Recognition,” Proc. First Int'l Symp. 3D Data Processing Visualization and Transmission, 2002.
[45] R. Tanawongsuwan and A. Bobick, “Modelling the Effects of Walking Speed on Appearance-Based Gait Recognition,” Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 783-790, 2004.
[46] G. Johansson, “Visual Perception of Biological Motion and a Model for Its Analysis,” Perception & Psychophysics, vol. 14, no. 2, pp. 201-211, 1973.
[47] E. Muybridge, The Human Figure in Motion. Dover Publications, 1901.
[48] D. Gavrilla, “The Visual Analysis of Human Movement: A Survey,” Computer Vision and Image Understanding, vol. 73, no. 1, pp. 82-98, Jan. 1999.
[49] E. Hoenkamp, “Perceptual Cues that Determine the Labelling of Human Gait,” J. Human Movement Studies, vol. 4, pp. 59-69, 1978.
[50] M. Murray, A. Drought, and R. Kory, “Walking Patterns of Normal Men,” J. Bone and Joint Surgery, vol. 46-A, no. A2, pp. 335-360, 1964.
[51] J. Cutting and L. Kozlowski, “Recognizing Friends by Their Walk: Gait Perception without Familiarity Cues,” Bull. Psychonomic Soc., vol. 9, no. 5, pp. 353-356, 1977.
[52] J. Cutting and D. Proffitt, “Gait Perception as an Example of How We May Perceive Events,” Intersensory Perception and Sensory Integration, 1981.
[53] G. Veres, L. Gordon, J. Carter, and M. Nixon, “What Image Information Is Important in Silhouette Based Gait Recognition?” Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 776-782, 2004.
[54] L. Rabiner and B. Juang, Fundamentals of Speech Recognition. Prentice Hall, 1993.
[55] A. Forner-Cordero, H. Koopman, and F. Van der Helm, “Describing Gait as a Sequence of States,” J. Biomechanics, to appear.
[56] J. Proakis and D. Manolakis, Digital Signal Processing: Principles, Algorithms and Applications, third ed. Prentice Hall, 1995.
[57] P. Brockwell and R. Davis, Time Series: Theory and Methods. Springer-Verlang, 1987.
[58] P. Overschee and B. Moor, “Subspace Algorithms for the Stochastic Identification Problem,” Automatica, vol. 29, pp. 649-660, 1993.
[59] S. Soatto, G. Doretto, and Y. Wu, “Dynamic Textures,” Proc. Int'l Conf. Computer Vision, vol. 2, pp. 439-446, 2001.
[60] G. Golub and C. Loan, Matrix Computations. Baltimore, Md.: The Johns Hopkins Univ. Press, 1989.
[61] K. Cock and D. Moor, “Subspace Angles and Distances between ARMA Models,” Proc. Int'l Symp. Math. Theory of Networks and Systems, 2000.
[62] A. Veeraraghavan, A. Roy-Chowdhury, and R. Chellappa, “Role of Shape and Kinematics in Human Movement Analysis,” Proc. Conf. Computer Vision and Pattern Recognition, 2004.
[63] A. Bobick and R. Tanawongsuwan, “Performance Analysis of Time-Distance Gait Parameters under Different Speeds,” Proc. Fourth Int'l Conf. Audio- and Video-Based Biometrie Person Authentication, June 2003.

Index Terms:
Index Terms- Shape, shape sequences, shape dynamics, comparison of shape sequences, gait recognition.
Ashok Veeraraghavan, Amit K. Roy-Chowdhury, Rama Chellappa, "Matching Shape Sequences in Video with Applications in Human Movement Analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1896-1909, Dec. 2005, doi:10.1109/TPAMI.2005.246
Usage of this product signifies your acceptance of the Terms of Use.