The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2013 vol.35)
pp: 1248-1262
V. I. Morariu , Dept. of Comput. Sci., Univ. of Maryland, College Park, MD, USA
D. Harwood , Dept. of Comput. Sci., Univ. of Maryland, College Park, MD, USA
L. S. Davis , Dept. of Comput. Sci., Univ. of Maryland, College Park, MD, USA
ABSTRACT
We describe a framework that leverages mixed probabilistic and deterministic networks and their AND/OR search space to efficiently find and track the hands and feet of multiple interacting humans in 2D from a single camera view. Our framework detects and tracks multiple people's heads, hands, and feet through partial or full occlusion; requires few constraints (does not require multiple views, high image resolution, knowledge of performed activities, or large training sets); and makes use of constraints and AND/OR Branch-and-Bound with lazy evaluation and carefully computed bounds to efficiently solve the complex network that results from the consideration of interperson occlusion. Our main contributions are: 1) a multiperson part-based formulation that emphasizes extremities and allows for the globally optimal solution to be obtained in each frame, and 2) an efficient and exact optimization scheme that relies on AND/OR Branch-and-Bound, lazy factor evaluation, and factor cost sensitive bound computation. We demonstrate our approach on three datasets: the public single person HumanEva dataset, outdoor sequences where multiple people interact in a group meeting scenario, and outdoor one-on-one basketball videos. The first dataset demonstrates that our framework achieves state-of-the-art performance in the single person setting, while the last two demonstrate robustness in the presence of partial and full occlusion and fast nontrivial motion.
INDEX TERMS
Extremities, Probabilistic logic, Pattern analysis, Training, Graphical models, Search problems,pictorial structures, Tracking, motion
CITATION
V. I. Morariu, D. Harwood, L. S. Davis, "Tracking People's Hands and Feet Using Mixed Network AND/OR Search", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 5, pp. 1248-1262, May 2013, doi:10.1109/TPAMI.2012.187
REFERENCES
[1] R. Dechter and R. Mateescu, "AND/OR Search Spaces for Graphical Models," Artificial Intelligence, vol. 171, pp. 73-106, 2007.
[2] R. Mateescu and R. Dechter, "Mixed Deterministic and Probabilistic Networks," Annals Math, and Artificial Intelligence, special issue: probabilistic relational learning, vol. 54, nos. 1-3, pp. 3-51, 2008.
[3] R. Marinescu and R. Dechter, "AND/OR Branch-and-Bound Search for Combinatorial Optimization in Graphical Models," Artificial Intelligence, vol. 173, nos. 16-17, pp. 1457-1491, 2009.
[4] C.-S. Lee and A. Elgammal, "Body Pose Tracking from Uncalibrated Camera Using Supervised Manifold Learning," Proc. Workshop Evaluation of Articulated Human Motion and Pose Estimation, 2006.
[5] R. Urtasun and T. Darrell, "Sparse Probabilistic Regression for Activity-Independent Human Pose Inference," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[6] P.F. Felzenszwalb and D.P. Huttenlocher, "Pictorial Structures for Object Recognition," Int'l J. Computer Vision, vol. 61, pp. 55-79, 2005.
[7] D. Ramanan, D. Forsyth, and A. Zisserman, "Tracking People by Learning Their Appearance," IEEE Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp. 65-81, Jan. 2007.
[8] E.B. Sudderth, M.I. Mandel, W.T. Freeman, and A.S. Willsky, "Distributed Occlusion Reasoning for Tracking with Nonparametric Belief Propagation," Proc. Advances in Neural Information Processing System, 2004.
[9] L. Sigal and M. Black, "Measure Locally, Reason Globally: Occlusion-Sensitive Articulated Pose Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[10] A. Gupta, A. Mittal, and L.S. Davis, "Constraint Integration for Efficient Multiview Pose Estimation with Self-Occlusions," IEEE Trans. Pattern Analysis Machine Intelligence, vol. 30, no. 3, pp. 493-506, Mar. 2008.
[11] H. Jiang and D. Martin, "Global Pose Estimation Using Non-Tree Models," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[12] X. Ren, A. Berg, and J. Malik, "Recovering Human Body Configurations Using Pairwise Constraints between Parts," Proc. IEEE Int'l Conf. Computer Vision, 2005.
[13] G. Hua, M.-H. Yang, and Y. Wu, "Learning to Estimate Human Pose with Data Driven Belief Propagation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[14] L. Karlinsky, M. Dinerstein, D. Harari, and S. Ullman, "The Chains Model for Detecting Parts by Their Context," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[15] H. Jiang, "Human Pose Estimation Using Consistent Max-Covering," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[16] T.-P. Tian and S. Sclaroff, "Fast Globally Optimal 2D Human Detection with Loopy Graph Models," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[17] M. Bergtholdt, J.H. Kappes, S. Schmidt, and C. Schnörr, "A Study of Parts-Based Object Class Detection Using Complete Graphs," Int'l J. Computer Vision, vol. 87, pp. 93-117, 2010.
[18] M. Andriluka, S. Roth, and B. Schiele, "People-Tracking-by-Detection and People-Detection-by-Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[19] S. Gammeter, A. Ess, T. Jäggli, K. Schindler, B. Leibe, and L.V. Gool, "Articulated Multi-Body Tracking under Egomotion," Proc. European Conf. Computer Vision, 2008.
[20] T. Zhao and R. Nevatia, "Tracking Multiple Humans in Complex Situations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1208-1221, Sept. 2004.
[21] S. Park and J.K. Aggarwal, "Simultaneous Tracking of Multiple Body Parts of Interacting Persons," Computer Vision and Image Understanding, vol. 102, pp. 1-21, 2006.
[22] T. Yu and Y. Wu, "Decentralized Multiple Target Tracking Using Netted Collaborative Autonomous Trackers," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[23] M. Eichner and V. Ferrari, "We Are Family: Joint Pose Estimation of Multiple Persons," Proc. European Conf. Computer Vision, 2010.
[24] L. Zhu, Y. Chen, Y. Lu, C. Lin, and A. Yuille, "Max Margin and/or Graph Learning for Parsing the Human Body," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[25] C.H. Lampert, M.B. Blaschko, and T. Hofmann, "Beyond Sliding Windows: Object Localization by Efficient Subwindow Search," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[26] V. Lempitsky, A. Blake, and C. Rother, "Image Segmentation by Branch-and-Mincut," Proc. European Conf. Computer Vision, 2008.
[27] R. Dechter, B. Bidyuk, R. Mateescu, and E. Rollon, "On the Power of Belief Propagation: A Constraint Propagation Perspective," Heuristics, Probabilities and Causality: A Tribute to Judea Pearl, College Publications, 2010.
[28] R. Dechter, Constraint Processing. Morgan Kaufmann, 2003.
[29] R. Dechter, "Bucket Elimination: A Unifying Framework for Reasoning," Artificial Intelligence, vol. 113, pp. 41-85, 1999.
[30] K. Kask and R. Dechter, "A General Scheme for Automatic Generation of Search Heuristics from Specification Dependencies," Artificial Intelligence, vol. 129, pp. 91-131, 2001.
[31] I. Haritaoglu, D. Harwood, and L. Davis, "W4: Real-Time Surveillance of People and Their Activities," IEEE Trans. Pattern Analysis Machine Intelligence, vol. 22, no. 8, pp. 809-830, Aug. 2000.
[32] R. Ronfard, C. Schmid, and B. Triggs, "Learning to Parse Pictures of People," Proc. European Conf. Computer Vision, 2002.
[33] W. Schwartz, A. Kembhavi, D. Harwood, and L. Davis, "Human Detection Using Partial Least Squares Analysis," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[34] C. Huang, B. Wu, and R. Nevatia, "Robust Object Tracking by Hierarchical Association of Detection Responses," Proc. European Conf. Computer Vision, 2008.
[35] M. Everingham, J. Sivic, and A. Zisserman, "'Hello! My Name is... Buffy'—Automatic Naming of Characters in TV Video," Proc. British Machine Vision Conf., pp. 889-908, 2006.
[36] Y. Li, C. Huang, and R. Nevatia, "Learning to Associate: Hybridboosted Multi-Target Tracker for Crowded Scene," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2953-2960, 2009.
[37] H. Sidenbladh and M.J. Black, "Learning Image Statistics for Bayesian Tracking," Proc. IEEE Int'l Conf. Computer Vision, 2001.
[38] K. Kim, T.H. Chalidabhongse, D. Harwood, and L.S. Davis, "Real-Time Foreground-Background Segmentation Using Codebook Model," Real-Time Imaging, vol. 11, pp. 172-185, 2005.
[39] A.S. Ogale and Y. Aloimonos, "A Roadmap to the Integration of Early Visual Modules," Int'l J. Computer Vision, vol. 72, pp. 9-25, 2007.
[40] D. Comaniciu and P. Meer, "Mean Shift Analysis and Applications," Proc. IEEE Int'l Conf. Computer Vision, 1999.
[41] V.I. Morariu, B.V. Srinivasan, V.C. Raykar, R. Duraiswami, and L.S. Davis, "Automatic Online Tuning for Fast Gaussian Summation," Proc. Advances in Neural Information Processing Systems, 2008.
[42] L. Sigal, A.O. Balan, and M.J. Black, "Humaneva: Synchronized Video and Motion Capture Data Set and Baseline Algorithm for Evaluation of Articulated Human Motion," Int'l J. Computer Vision, vol. 87, nos. 1/2, pp. 4-27, 2010.
[43] J. Martinez del Rincon, J. Nebel, D. Makris, and C. Orrite, "Tracking Human Body Parts Using Particle Filters Constrained by Human Biomechanics," Proc. British Machine Vision Conf., 2008.
[44] R. Poppe, "Evaluating Example-Based Pose Estimation: Experiments on the Humaneva Sets," Proc. Workshop Evaluation of Articulated Human Motion and Pose Estimation, 2007.
[45] N. Howe, "Evaluating Lookup-Based Monocular Human Pose Tracking on the Humaneva Test Data," Proc. Workshop Evaluation of Articulated Human Motion and Pose Estimation, 2006.
[46] C. Yanover and Y. Weiss, "Finding the m Most Probable Configurations Using Loopy Belief Propagation," Proc. Advances in Neural Information Processing Systems, 2004.
[47] J.M. Mooij, "libDAI: A Free and Open Source C++ Library for Discrete Approximate Inference in Graphical Models," J. Machine Learning Research, vol. 11, pp. 2169-2173, Aug. 2010.
[48] W.S. Cleveland and S.J. Devlin, "Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting," J. Am. Statistical Assoc., vol. 83, no. 403, pp. 596-610, Sept. 1988.
[49] P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan, "Object Detection with Discriminatively Trained Part Based Models," IEEE Trans. Pattern Analysis Machine Intelligence, vol. 32, no. 9, pp. 1627-1645, Sept. 2010.
[50] W.R. Schwartz, A. Kembhavi, D. Harwood, and L.S. Davis, "Human Detection Using Partial Least Squares Analysis," Proc. IEEE Int'l Conf. Computer Vision, 2009.
94 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool