The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan. (2013 vol.35)
pp: 52-65
Marek Vondrak , Dept. of Comput. Sci., Brown Univ., Providence, RI, USA
L. Sigal , Disney Res., Pittsburgh, PA, USA
O. C. Jenkins , Dept. of Comput. Sci., Brown Univ., Providence, RI, USA
ABSTRACT
We propose a simulation-based dynamical motion prior for tracking human motion from video in presence of physical ground-person interactions. Most tracking approaches to date have focused on efficient inference algorithms and/or learning of prior kinematic motion models; however, few can explicitly account for the physical plausibility of recovered motion. Here, we aim to recover physically plausible motion of a single articulated human subject. Toward this end, we propose a full-body 3D physical simulation-based prior that explicitly incorporates a model of human dynamics into the Bayesian filtering framework. We consider the motion of the subject to be generated by a feedback “control loop” in which Newtonian physics approximates the rigid-body motion dynamics of the human and the environment through the application and integration of interaction forces, motor forces, and gravity. Interaction forces prevent physically impossible hypotheses, enable more appropriate reactions to the environment (e.g., ground contacts), and are produced from detected human-environment collisions. Motor forces actuate the body, ensure that proposed pose transitions are physically feasible, and are generated using a motion controller. For efficient inference in the resulting high-dimensional state space, we utilize an exemplar-based control strategy that reduces the effective search space of motor forces. As a result, we are able to recover physically plausible motion of human subjects from monocular and multiview video. We show, both quantitatively and qualitatively, that our approach performs favorably with respect to Bayesian filtering methods with standard motion priors.
INDEX TERMS
Kinematics, Tracking, Humans, Dynamics, Joints, Biological system modeling, Trajectory,particle filtering, Articulated tracking, human pose tracking, human motion, physical simulation, physics-based priors, Bayesian filtering
CITATION
Marek Vondrak, L. Sigal, O. C. Jenkins, "Dynamical Simulation Priors for Human Motion Tracking", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 1, pp. 52-65, Jan. 2013, doi:10.1109/TPAMI.2012.61
REFERENCES
[1] A. Balan, L. Sigal, M.J. Black, J. Davis, and H. Haussecker, "Detailed Human Shape and Pose from Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[2] A. Balan, L. Sigal, and M.J. Black, "A Quantitative Evaluation of Video-Based 3D Person Tracking," Proc. IEEE Workshop Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 349-356, Oct. 2005.
[3] J. Bandouch, F. Engstler, and M. Beetz, "Accurate Human Motion Capture Using an Ergonomics-Based Anthropometric Human Model," Proc. Int'l Conf. Articulated Motion and Deformable Objects, 2008.
[4] D. Baraff, "Linear-Time Dynamics Using Lagrange Multipliers," Proc. Ann. Conf. Computer Graphics and Interactive Techniques, pp. 137-146, 1996.
[5] L. Bo, C. Sminchisescu, A. Kanaujia, and D. Metaxas, "Fast Algorithms for Large Scale Conditional 3D Prediction," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[6] M. Brubaker, L. Sigal, and D.J. Fleet, "Estimating Contact Dynamics," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[7] M. Brubaker, D.J. Fleet, and A. Hertzmann, "Physics-Based Person Tracking Using Simplified Lower-Body dynamics," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[8] M. Brubaker and D.J. Fleet, "The Kneed Walker for Human Pose Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[9] J. Chai and J. Hodgins, "Constraint-Based Motion Optimization Using A Statistical Dynamic Model," ACM Trans. Graphics (Siggraph), vol. 26, no. 3,article 8, 2007.
[10] N. Chakraborty, S. Berard, S. Akella, and J.C. Trinkle, "A Fully Implicit Time-Stepping Method for Multibody Systems with Intermittent Contact," Proc. Robotics: Science and Systems, 2007.
[11] Y. Choi, B.-J. You, and S.-R. Oh, "On the Stability of Indirect ZMP Controller for Biped Robot Systems," Proc. IEEE/RSJ Int'l Conf. Intelligent Robots and Systems, vol. 2, pp. 1966-1971, 2004.
[12] P. DeVita and T. Hortobagyi, "Age Causes a Redistribution of Joint Torques and Powers during Gait," J. Applied Physiology, vol. 88, pp. 1804-1811, 2000.
[13] J. Deutscher and I. Reid, "Articulated Body Motion Capture by Stochastic Search," Int'l J. Computer Vision, vol. 61, no. 2, pp. 185-205, 2004.
[14] A. Doucet, N.de Freitas, and N. Gordon, "Sequential Monte Carlo Methods in Practice," Statistics for Eng. and Information Sciences, Springer Verlag, 2001.
[15] P. Faloutsos, M. van de Panne, and D. Terzopoulos, "Composable Controllers for Physics-Based Character Animation," Proc. 28th Ann. Conf. Computer Graphics and Interactive Techniques (Siggraph), 2001.
[16] A.C. Fang and N.S. Pollard, "Efficient Synthesis of Physically Valid Human Motion," ACM Trans. Graphics, vol. 22, no. 3, pp. 417-426, 2003.
[17] D.A. Forsyth, O. Arikan, L. Ikemoto, J. O'Brien, and D. Ramanan, "Computational Studies of Human Motion: Part 1, Tracking and Motion Synthesis," Foundations and Trends in Computer Graphics and Vision, vol. 1, pp. 77-254, July 2006.
[18] K. Grauman, G. Shakhnarovich, and T. Darrell, "Inferring 3D Structure with a Statistical Image-Based Shape Model," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[19] J. Hodgins, W. Wooten, D. Brogan, and J. O'Brien, "Animating Human Athletics," Proc. ACM Siggraph, pp. 71-78, 1995.
[20] O.C. Jenkins and M.J. Mataric, "Performance-Derived Behavior Vocabularies: Data-Driven Acqusition of Skills from Motion," Int'l J. Humanoid Robotics, vol. 1, no. 2 237-288, 2004.
[21] E. Kokkevis, "Practical Physics for Articulated Characters," Proc. Game Developers Conf., 2004.
[22] A.D. Kuo, "A Least-Squares Estimation Approach to Improving the Precision of Inverse Dynamics Computations," J. Biomechanical Eng., vol. 120, no. 1, pp. 148-159, 1998.
[23] C. Lee and A. Elgammal, "Modeling View and Posture Manifolds for Tracking," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[24] R. Li, T.-P. Tian, S. Sclaroff, and M.-H. Yang, "3D Human Motion Tracking with a Coordinated Mixture of Factor Analyzers," Int'l J. Computer Vision, vol. 87, pp. 170-190, 2010.
[25] R. Li, T. Tian, and S. Sclaroff, "Simultaneous Learning of Nonlinear Manifold and Dynamical Models for High-Dimensional Time Series," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[26] Z. Lu, M. Carreira-Perpinan, and C. Sminchisescu, "People Tracking with the Laplacian Eigenmaps Latent Variable Model," Proc. Neural Information Processing Systems, 2007.
[27] J. McCann, N. Pollard, and S. Srinivasa, "Physics-Based Motion Retiming," Proc. ACM Siggraph/Eurographics Symp. Computer Animation, 2006.
[28] P. Michel, J. Chestnutt, S. Kagami, K. Nishiwaki, J. Kuffner, and T. Kanade, "Online Environment Reconstruction for Biped Navigation," Proc. IEEE Int'l Conf. Robotics and Automation, pp. 3089-3094, 2006.
[29] T. Moeslund, A. Hilton, and V. Kruger, "A Survey of Advances in Vision-Based Human Motion Capture and Analysis," Int'l J. Computer Vision and Image Understanding, vol. 104, no. 2, pp. 90-126, 2006.
[30] D. Metaxas and D. Terzopoulos, "Shape and Nonrigid Motion Estimation through Physics-Based Synthesis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 580-591, June 1993.
[31] Y. Nakamura and K. Yamane, "Dynamics Computation of Structure-Varying Kinematic Chains and Its Application to Human Figures," IEEE Trans. Robotics and Automation, vol. 16, no. 2, pp. 124-134, Apr. 2000.
[32] V. Pavlovic, J. Rehg, T.J. Cham, and K. Murphy, "A Dynamic Bayesian Network Approach to Figure Tracking Using Learned Dynamic Models," Proc. IEEE Int'l Conf. Computer Vision, pp. 94-101, 1999.
[33] Z. Popovic and A. Witkin, "Physically Based Motion Transformation," Proc. ACM Siggraph, 1999.
[34] R. Poppe, "Vision-Based Human Motion Analysis: An Overview," Int'l J. Computer Vision and Image Understanding, vol. 108, nos. 1/2, pp. 4-18, 2007.
[35] C.E. Rasmussen and C. Williams, Gaussian Processes for Machine Learning. MIT Press, 2006.
[36] R. Riemer and E.T. Hsiao-Wecksler, "Improving Joint Torque Calculations: Optimization-Based Inverse Dynamics to Reduce the Effect of Motion Errors," J. Biomechanics, vol. 41, no. 7, pp. 1503-1509, 2008.
[37] B. Rosenhahn, C. Schmaltz, T. Brox, and H.-P. Seidel, "Staying Well Grounded in Markerless Motion Capture," Proc. 30th DAGM Symp. Pattern Recognition, 2008.
[38] B. Rosenhahn, C. Schmaltz, T. Brox, J. Weickert, D. Cremers, and H.-P. Seidel, "Markerless Motion Capture of Man-Machine Interaction," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[39] A. Safonova, J. Hodgins, and N. Pollard, "Synthesizing Physically Realistic Human Motion in Low-Dimensional, Behavior-Specific Spaces," ACM Trans. Graphics (SIGGRAPH), vol. 23, no. 3, pp. 514-521, 2004.
[40] G. Shakhnarovich, P. Viola, and T. Darrell, "Fast Pose Estimation with Parameter Sensitive Hashing," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 750-757, 2003.
[41] L. Sigal and M.J. Black, "HumanEva: Synchronized Video and Motion Capture Data Set for Evaluation of Articulated Human Motion," Technical Report CS-06-08, Brown Univ., 2006.
[42] B. Siciliano and O. Khatib, Springer Handbook of Robotics. Springer, 2008.
[43] H. Sidenbladh, M.J. Black, and L. Sigal, "Implicit Probabilistic Models of Human Motion for Synthesis and Tracking," Proc. European Conf. Computer Vision, vol. 1, pp. 784-800, 2002.
[44] H. Sidenbladh and M.J. Black, "Learning Image Statistics for Bayesian Tracking," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 709-716, 2001.
[45] C. Sminchisescu, A. Kanaujia, Z. Li, and D. Metaxas, "Discriminative Density Propagation for 3D Human Motion Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 390-397, 2005.
[46] C. Sminchisescu and A. Jepson, "Generative Modeling for Continuous Non-Linearly Embedded Visual Inference," Proc. Int'l Conf. Machine Learning, pp. 759-766, 2004.
[47] M.W. Spong, S. Hutchinson, and M. Vidyasagar, Robot Modeling and Control. Wiley, 2006.
[48] B. Stephens, "Humanoid Push Recovery," Proc. IEEE-RAS Seventh Int'l Conf. Humanoid Robots, 2007.
[49] R. Urtasun and T. Darrell, "Local Probabilistic Regression for Activity-Independent Human Pose Inference," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[50] R. Urtasun, D.J. Fleet, A. Geiger, J. Popovic, T. Darrell, and N.D. Lawrence, "Topologically-Constrained Latent Variable Models," Proc. Int'l Conf. Machine Learning, 2008.
[51] R. Urtasun, D. Fleet, and P. Fua, "Gaussian Process Dynamical Models for 3D People Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[52] R. Urtasun, D.J. Fleet, A. Hertzmann, and P. Fua., "Priors for People Tracking from Small Training Sets," Proc. IEEE Int'l Conf. Computer Vision, 2005.
[53] M. Vondrak, L. Sigal, and O.C. Jenkins, "Dynamics and Control of Multibody Systems," Motion Control, IN-TECH, 2009.
[54] M. Vondrak, L. Sigal, and O.C. Jenkins, "Physical Simulation for Probabilistic Motion Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[55] D.A. Winter, Biomechanics and Motor Control of Human Movement. Wiley, 2005.
[56] C.R. Wren and A. Pentland, "Dynamic Models of Human Motion," Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 1998.
[57] P. Wrotek, O. Jenkins, and M. McGuire, "Dynamo: Dynamic Data-Driven Character Control with Adjustable Balance," Proc. ACM Siggraph Symp. Videogame, 2006.
[58] X. Xu and B. Li, "Learning Motion Correlation for Tracking Articulated Human Body with a Rao-Blackwellised Particle Filter," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[59] K. Yamane and Y. Nakamura, "Robot Kinematics and Dynamics for Modeling the Human Body," Proc. Int'l Symp. Robotics Research, 2007.
[60] K. Yamane and Y. Nakamura, "Automatic Scheduling for Parallel Forward Dynamics Computation of Open Kinematic Chains," Proc. Robotics: Science and Systems, 2007.
[61] K. Yin, S. Coros, P. Beaudoin, and M. van de Panne, "Continuation Methods for Adapting Simulated Skills," Proc. ACM Siggraph, 2008.
[62] K. Yin, K. Loken, and M. van de Panne, "SIMBICON: Simple Biped Locomotion Control," Proc. ACM Siggraph, 2007.
[63] V. Zordan, A. Majkowska, B. Chiu, and M. Fast, "Dynamic Response for Motion Capture Animation," Proc. ACM Siggraph, 2005.
[64] http:/crisis.sourceforge.net/, 2012.
105 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool