| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
Actions as Space-Time Shapes
December 2007 (vol. 29 no. 12)
pp. 2247-2253
Human action in video sequences can be seen as silhouettes of a moving torso and protruding limbs undergoing articulated motion. We regard human actions as three- imensional shapes induced by the silhouettes in the spacetime volume. We adopt a recent approach [14] for analyzing 2D shapes and generalize it to deal with volumetric space-time action shapes. Our method utilizes properties of the solution to the Poisson equation to extract spacetime features such as local space-time saliency, action dynamics, shape structure and orientation. We show that these features are useful for action recognition, detection and clustering. The method is fast, does not require video alignment and is applicable in (but not limited to) many scenarios where the background is known. Moreover, we demonstrate the robustness of our method to partial occlusions, non-rigid deformations, significant changes in scale and viewpoint, high irregularities in the performance of an action, and low quality video.
[1] 2247 S. Belongie, J. Malik, and J. Puzicha, “Shape Matching and Object Recognition Using Shape Contexts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 509-522, Apr. 2002.[2] P.J. Besl and R.C. Jain, “Invariant Surface Characteristics for 3D Object Recognition in Range Images,” Computer Vision, Graphics, and Image Processing, vol. 33, no. 1, pp. 33-80, 1986.[3] M.J. Black, “Explaining Optical Flow Events with Parameterized Spatio-Temporal Models,” Computer Vision and Pattern Recognition, vol. 1, pp. 1326-1332, 1999.[4] M. Blank, L. Gorelick, E. Shechtman, M. Irani, and R. Basri, “Actions as Space-Time Shapes,” Proc. Int'l Conf. Computer Vision, pp. 1395-1402, 2005.[5] H. Blum, “A Transformation for Extracting New Descriptors of Shape,” Models for the Perception of Speech and Visual Form, Proc. Symp., pp. 362-380, 1967.[6] A. Bobick and J. Davis, “The Recognition of Human Movement Using Temporal Templates,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 257-267, Mar. 2001.[7] C. Bregler, “Learning and Recognizing Human Dynamics in Video Sequences,” Proc. Computer Vision and Pattern Recognition, June 1997.[8] S. Carlsson, “Order Structure, Correspondence and Shape Based Categories,” Proc. Int'l Workshop Shape, Contour, and Grouping, p. 1681, 1999.[9] S. Carlsson and J. Sullivan, “Action Recognition by Shape Matching to Key Frames,” Proc. Workshop Models versus Exemplars in Computer Vision, Dec. 2001.[10] O. Chomat and J.L. Crowley, “Probabilistic Sensor for the Perception of Activities,” Proc. European Conf. Computer Vision, 2000.[11] A.A. Efros, A.C. Berg, G. Mori, and J. Malik, “Recognizing Action at a Distance,” Proc. Int'l Conf. Computer Vision, Oct. 2003.[12] T. Fan, G. Medioni, and A. Nevatia, “Matching 3-D Objects Using Surface Descriptions,” Proc. IEEE Int'l Conf. Robotics and Automation, vol. 3, no. 24-29, pp. 1400-1406, 1988.[13] R. Goldenberg, R. Kimmel, E. Rivlin, and M. Rudzsky, “Behavior Classification by Eigendecomposition of Periodic Motions,” Pattern Recognition, vol. 38, no. 7, pp. 1033-1043, 2005.[14] L. Gorelick, M. Galun, E. Sharon, A. Brandt, and R. Basri, “Shape Representation and Classification Using the Poisson Equation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 12, Dec. 2006.[15] S.X. Ju, M.J. Black, and Y. Yacoob, “Cardboard People: A Parametrized Model of Aticulated Image Motion,” Proc. Second Int'l Conf. Automatic Face and Gesture Recognition, pp. 38-44, Oct. 1996.[16] Y. Ke, R. Sukthankar, and M. Hebert, “Efficient Visual Event Detection Using Volumetric Features,” Proc. Int'l Conf. Computer Vision, pp. 166-173, 2005.[17] I. Laptev and T. Lindeberg, “Space-Time Interest Points,” Proc. Int'l Conf. Computer Vision, 2003.[18] G. Medioni and C. Tang, “Tensor Voting: Theory and Applications,” Proc. 12th Congres Francophone AFRIF-AFIA de Reconnaissance des Formes et Intelligence Artificielle, 2000.[19] A. Ng, M. Jordan, and Y. Weiss, “On Spectral Clustering: Analysis and an Algorithm,” Proc. Advances in Neural Information Processing Systems 14, pp.849-856, 2001.[20] S.A. Niyogi and E.H. Adelson, “Analyzing and Recognizing Walking Figures in xyt,” Proc. Computer Vision and Pattern Recognition, June 1994.[21] R. Polana and R.C. Nelson, “Detection and Recognition of Periodic, Nonrigid Motion,” Int'l J. Computer Vision, vol. 23, no. 3, 1997.[22] E. Rivlin, S. Dickinson, and A. Rosenfeld, “Recognition by Functional Parts,” Proc. Computer Vision and Pattern Recognition, pp. 267-274, 1994.[23] T. Sebastian, P. Klein, and B. Kimia, “Shock-Based Indexing into Large Shape Databases,” Proc. European Conf. Computer Vision, vol. 3, pp. 731-746, 2002.[24] S. Seitz and C. Dyer, “View-Invariant Analysis of Cyclic Motion,” Int'l J. Computer Vision, vol. 25, no. 3, pp. 231-251, Dec. 1997.[25] E. Shechtman and M. Irani, “Space-Time Behavior Based Correlation,” Proc. Computer Vision and Pattern Recognition, June 2005.[26] K. Siddiqi, A. Shokoufandeh, S.J. Dickinson, and S.W. Zucker, “Shock Graphs and Shape Matching,” Proc. IEEE Int'l Conf. Computer Vision, p. 222, 1998.[27] http://www.wisdom.weizmann.ac.il/~vision SpaceTimeActions.html, 2005.[28] J. Tangelder and R. Veltkamp, “A Survey of Content Based 3D Shape Retrieval Methods,” Proc. Shape Modeling Int'l, pp. 145-156, 2004.[29] U. Trottenberg, C. Oosterlee, and A. Schuller, Multigrid. Academic Press, 2001.[30] U. Weidenbacher, P. Bayerl, H. Neumann, and R. Fleming, “Sketching Shiny Surfaces: 3D Shape Extraction and Depiction of Specular Surfaces,” ACM Trans. Applied Perception, vol. 3, no. 3, pp. 262-285, 2006.[31] Y. Yacoob and M.J. Black, “Parametrized Modeling and Recognition of Activities,” Computer Vision and Image Understanding, vol. 73, no. 2, pp. 232-247, 1999.[32] A. Yilmaz and M. Shah, “Actions Sketch: A Novel Action Representation,” Computer Vision and Pattern Recognition, vol. 1, pp. 984-989, 2005.[33] L. Zelnik-Manor and M. Irani, “Event-Based Analysis of Video,” Computer Vision and Pattern Recognition, pp. 123-130, Sept. 2001.
Index Terms:
Action representation, action recognition, space-time analysis, shape analysis, poisson equation
Citation:
Lena Gorelick, Moshe Blank, Eli Shechtman, Michal Irani, Ronen Basri, "Actions as Space-Time Shapes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2247-2253, June 2007, doi:10.1109/TPAMI.2007.70711
