The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2013 vol.35)
pp: 1577-1591
Wongun Choi , Dept. of Electr. & Comput. Eng., Univ. of Michigan, Ann Arbor, MI, USA
C. Pantofaru , Willow Garage, Inc., Menlo Park, CA, USA
S. Savarese , Dept. of Electr. & Comput. Eng., Univ. of Michigan, Ann Arbor, MI, USA
ABSTRACT
In this paper, we present a general framework for tracking multiple, possibly interacting, people from a mobile vision platform. To determine all of the trajectories robustly and in a 3D coordinate system, we estimate both the camera's ego-motion and the people's paths within a single coherent framework. The tracking problem is framed as finding the MAP solution of a posterior probability, and is solved using the reversible jump Markov chain Monte Carlo (RJ-MCMC) particle filtering method. We evaluate our system on challenging datasets taken from moving cameras, including an outdoor street scene video dataset, as well as an indoor RGB-D dataset collected in an office. Experimental evidence shows that the proposed method can robustly estimate a camera's motion from dynamic scenes and stably track people who are moving independently or interacting.
INDEX TERMS
Cameras, Target tracking, Detectors, Face, Skin, Trajectory,RJ-MCMC particle filtering, Multitarget tracking, person detection, people tracking
CITATION
Wongun Choi, C. Pantofaru, S. Savarese, "A General Framework for Tracking Multiple People from a Moving Camera", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 7, pp. 1577-1591, July 2013, doi:10.1109/TPAMI.2012.248
REFERENCES
[1] OpenCV, The Open Source Computer Vision Library, http://opencv.willowgarage.comwiki/, 2012.
[2] ROS, The Robot Operating System, http:/www.ros.org/, 2012.
[3] M. Andriluka, S. Roth, and B. Schiele, "People-Tracking-by-Detection and People-Detection-by-Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[4] M.S. Arulampalam, S. Maskell, and N. Gordon, "A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking," IEEE Trans. Signal Processing, vol. 50, pp. 174-188, Feb. 2002.
[5] S. Avidan, "Ensemble Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 261-271, Feb. 2007.
[6] H. Bay, A. Ess, T. Tuytelaars, and L.V. Gool, "Surf: Speeded Up Robust Features," Computer Vision and Image Understanding, vol. 110, pp. 346-359, 2008.
[7] C. Bibby and I. Reid, "Robust Real-Time Visual Tracking Using Pixel-Wise Posteriors," Proc. 10th European Conf. Computer Vision, 2008.
[8] M.D. Breitenstein, F. Reichlin, B. Leibe, E. Koller-Meier, and L.V. Gool, "Robust Tracking-by-Detection Using a Detector Confidence Particle Filter," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[9] W. Choi and S. Savarese, "Multiple Target Tracking in World Coordinate with Single, Minimally Calibrated Camera," Proc. European Conf. Computer Vision, 2010.
[10] D. Comaniciu and P. Meer, "Mean Shift: A Robust Approach Toward Feature Space Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[11] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[12] A. Ess, B. Leibe, K. Schindler, and L. van Gool, "A Mobile Vision System for Robust Multi-Person Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[13] A. Ess, B. Leibe, K. Schindler, and L. van Gool, "Robust Multi-Person Tracking from a Mobile Platform," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 10, pp. 1831-1846, Oct. 2009.
[14] M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn, and A. Zisserman, "The Pascal Visual Object Classes (VOC) Challenge," Int'l J. Computer Vision, vol. 88, no. 2, pp. 303-338, June 2010.
[15] P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan, "Object Detection with Discriminatively Trained Part Based Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1627-1645, Sept. 2010.
[16] V. Ferrari, M. Marin-Jimenez, and A. Zisserman, "Progressive Search Space Reduction for Human Pose Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[17] M.A. Fischler and R.C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Comm. ACM, vol. 24, no. 6, pp. 381-395, 1981.
[18] J. Gower and G. Dijksterhuis, Procrustes Problems. Oxford Univ. Press, 2004.
[19] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[20] D. Hoiem, A.A. Efros, and M. Herbert, "Putting Objects in Perspective," Int'l J. Computer Vision, vol. 80, pp. 3-15, 2008.
[21] S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison, and A. Fitzgibbon, "Kinectfusion: Real-Time 3D Reconstruction and Interaction Using a Moving Depth Camera," Proc. ACM Symp. User Interface Software and Technology, 2011.
[22] M. Jones and J. Rehg, "Statistical Color Models with Application to Skin Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[23] J. Kammerl, "Octree Point Cloud Compression in PCL," http://pointclouds.org/newscompressing-point-clouds.html , 2011.
[24] Z. Khan, T. Balch, and F. Dellaert, "MCMC-Based Particle Filtering for Tracking a Variable Number of Interacting Targets," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1805-1819, Nov. 2005.
[25] H.W. Kuhn, "The Hungarian Method for the Assignment Problem," Naval Research Logistics Quarterly, vol. 2, pp. 83-97, 1955.
[26] S. Kwak, W. Nam, B. Han, and J. Han, "Learning Occlusion with Likelihoods," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[27] B. Leibe, A. Leonardis, and B. Schiele, "Combined Object Categorization and Segmentation with an Implicit Shape Model," Proc. ECCV Statistical Learning in Computer Vision, 2004.
[28] R. Lienhart and J. Maydt, "An Extended Set of Haar-Like Features for Rapid Object Detection," Proc. Int'l Conf. Image Processing, 2002.
[29] Microsoft Corp, "Kinect for XBOX," http:/www.xbox.com, 2012.
[30] P. Minvielle, A. Marrs, S. Maskell, and A. Doucet, "Joint Target Tracking and Identification-Part I: Sequential Monte Carlo Model-Based Approaches," Proc. Int'l Conf. Information Fusion, 2005.
[31] S. Pellegrini, A. Ess, K. Schindler, and L. van Gool, "You'll Never Walk Alone: Modeling Social Behavior for Multi-Target Tracking," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[32] H. Pirsiavash, D. Ramanan, and C. Fowlkes, "Globally-Optimal Greedy Algorithms for Tracking a Variable Number of Objects," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[33] PrimeSense, "NITE Natural Interaction Middleware," http://www.primesense.com?p=515, 2012.
[34] D. Ramanan, D. Forsyth, and A. Zisserman, "Tracking People by Learning their Appearance," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp. 64-81, Jan. 2007.
[35] R.B. Rusu and S. Cousins, "3D Is Here: Point Cloud Library (PCL)," Proc. Int'l Conf. Robotics and Automation, May 2011.
[36] P. Scovanner and M. Tappen, "Learning Pedestrian Dynamics from the Real World," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[37] H.B. Shitrit, J. Berclaz, F. Fleuret, and P. Fua, "Tracking Multiple People under Global Appearance Constraints," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[38] C. Tomasi and T. Kanade, "Detection and Tracking of Point Features," technical report, Carnegie Mellon Univ., 1991.
[39] O. Tuzel, F. Porikli, and P. Meer, "Human Detection via Classification on Riemannian Manifolds," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[40] P. Viola and M. Jones, "Robust Real-Time Face Detection," Int'l J. Computer Vision, vol. 57, no. 2, pp. 137-154, 2003.
[41] P. Viola, M. Jones, and D. Snow, "Detecting Pedestrians Using Patterns of Motion and Appearance," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[42] C. Wojek, S. Walk, S. Roth, and B. Schiele, "Monocular 3D Scene Understanding with Explicit Occlusion Reasoning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[43] C. Wojek, S. Walk, and B. Schiele, "Multi-Cue Onboard Pedestrian Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[44] B. Wu and R. Nevatia, "Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet Based Part Detectors," Int'l J. Computer Vision, vol. 75, pp. 247-266, 2007.
[45] J. Xing, H. Ai, and S. Lao, "Multi-Object Tracking through Occlusions by Local Tracklets Filtering and Global Tracklets Association with Detection Responses," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[46] L. Zhang, Y. Li, and R. Nevatia, "Global Data Association for Multi-Object Tracking Using Network Flows," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
27 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool