The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.09 - September (2011 vol.33)
pp: 1820-1833
Michael D. Breitenstein , ETH Zurich, Zurich
Fabian Reichlin , LiberoVision AG, Zurich
Bastian Leibe , RWTH Aachen University, Aachen
Esther Koller-Meier , ETH Zurich, Zurich
Luc Van Gool , ETH Zurich, Zurich and ESAT-PSI/IBBT, KU Leuven
ABSTRACT
In this paper, we address the problem of automatically detecting and tracking a variable number of persons in complex scenes using a monocular, potentially moving, uncalibrated camera. We propose a novel approach for multiperson tracking-by-detection in a particle filtering framework. In addition to final high-confidence detections, our algorithm uses the continuous confidence of pedestrian detectors and online-trained, instance-specific classifiers as a graded observation model. Thus, generic object category knowledge is complemented by instance-specific information. The main contribution of this paper is to explore how these unreliable information sources can be used for robust multiperson tracking. The algorithm detects and tracks a large number of dynamically moving people in complex scenes with occlusions, does not rely on background modeling, requires no camera or ground plane calibration, and only makes use of information from the past. Hence, it imposes very few restrictions and is suitable for online applications. Our experiments show that the method yields good tracking performance in a large variety of highly dynamic scenarios, such as typical surveillance videos, webcam footage, or sports sequences. We demonstrate that our algorithm outperforms other methods that rely on additional information. Furthermore, we analyze the influence of different algorithm components on the robustness.
INDEX TERMS
Multi-object tracking, tracking-by-detection, detector confidence particle filter, pedestrian detection, particle filtering, sequential Monte Carlo estimation, online learning, detector confidence, surveillance, sports analysis, traffic safety.
CITATION
Michael D. Breitenstein, Fabian Reichlin, Bastian Leibe, Esther Koller-Meier, Luc Van Gool, "Online Multiperson Tracking-by-Detection from a Single, Uncalibrated Camera", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 9, pp. 1820-1833, September 2011, doi:10.1109/TPAMI.2010.232
REFERENCES
[1] M. Andriluka, S. Roth, and B. Schiele, "People-Tracking-by-Detection and People-Detection-by-Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[2] S. Avidan, "Ensemble Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 261-271, Feb. 2007.
[3] J. Berclaz, F. Fleuret, and P. Fua, "Robust People Tracking with Global Trajectory Optimization," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[4] K. Bernardin and R. Stiefelhagen, "Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics," J. Image and Video Processing, vol. 2008, pp. 1-10, 2008.
[5] M.D. Breitenstein, F. Reichlin, B. Leibe, E. Koller-Meier, and L. Van Gool, "Markovian Tracking-by-Detection from a Single, Uncalibrated Camera," Proc. IEEE Workshop Performance Evaluation of Tracking and Surveillance, 2009.
[6] M.D. Breitenstein, F. Reichlin, B. Leibe, E. Koller-Meier, and L. Van Gool, "Robust Tracking-by-Detection Using a Detector Confidence Particle Filter," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[7] Y. Cai, N. de Freitas, and J.J. Little, "Robust Visual Tracking for Multiple Targets," Proc. European Conf. Computer Vision, 2006.
[8] R. Choudhury, C. Schmid, and K. Mikolajczyk, "Face Detection and Tracking in a Video by Propagating Detection Probabilities," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1215-1228, Oct. 2003.
[9] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2005.
[10] A. Doucet, N. De Freitas, and N. Gordon, Sequential Monte Carlo Methods in Practice. Springer, 2001.
[11] A. Ess, B. Leibe, K. Schindler, and L. Van Gool, "Robust Multi-Person Tracking from a Mobile Platform," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 10, pp. 1831-1846, Oct. 2009.
[12] J. Ferryman, IEEE Workshop Performance Evaluation of Tracking and Surveillance, 2009.
[13] T. Fortmann, Y. Bar Shalom, and M. Scheffe, "Sonar Tracking of Multiple Targets Using Joint Probabilistic Data Association," IEEE J. Oceanic Eng., vol. OE-8, no. 3, pp. 173-184, July 1983.
[14] S. Gammeter, A. Ess, T. Jaeggli, K. Schindler, B. Leibe, and L. Van Gool, "Articulated Multibody Tracking under Egomotion," Proc. European Conf. Computer Vision, 2008.
[15] J. Giebel, D. Gavrila, and C. Schnörr, "A Bayesian Framework for Multi-Cue 3D Object Tracking," Proc. European Conf. Computer Vision, 2004.
[16] N. Gordon, D. Salmond, and A. Smith, "Novel Approach to Nonlinear/Non-Gaussian Bayesian State Estimation," IEE Proc. F Radar Signal Processing, vol. 14, no. 2, pp. 107-113, Apr. 1993.
[17] H. Grabner and H. Bischof, "On-Line Boosting and Vision," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[18] C. Huang, B. Wu, and R. Nevatia, "Robust Object Tracking by Hierarchical Association of Detection Responses," Proc. European Conf. Computer Vision, 2008.
[19] M. Isard and A. Blake, "Condensation-Conditional Density Propagation for Visual Tracking," Int'l J. Computer Vision, vol. 29, no. 1, pp. 5-28, 1998.
[20] S.M. Khan and M. Shah, "A Multiview Approach to Tracking People in Crowded Scenes Using a Planar Homography Constraint," Proc. European Conf. Computer Vision, 2006.
[21] Z. Khan, T. Balch, and F. Dellaert, "MCMC-Based Particle Filtering for Tracking a Variable Number of Interacting Targets," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1805-1918, Nov. 2005.
[22] H. Kuhn, "The Hungarian Method for the Assignment Problem," Naval Research Logistics Quaterly, vol. 2, pp. 83-87, 1955.
[23] C.-H. Kuo, C. Huang, and R. Nevatia, "Multi-Target Tracking by On-Line Learned Discriminative Appearance Models," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[24] O. Lanz, "Approximate Bayesian Multibody Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1436-1449, Sept. 2006.
[25] B. Leibe, A. Leonardis, and B. Schiele, "Robust Object Detection with Interleaved Categorization and Segmentation," Int'l J. Computer Vision, vol. 77, nos. 1-3, pp. 259-289, 2008.
[26] B. Leibe, K. Schindler, N. Cornelis, and L.V. Gool, "Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1683-1698, Oct. 2008.
[27] Y. Li, H. Ai, C. Huang, and S. Lao, "Robust Head Tracking Based on a Multi-State Particle Filter," Proc. IEEE Int'l Conf. Face and Gesture Recognition, 2006.
[28] Y. Li, H. Ai, T. Yamashita, S. Lao, and M. Kawade, "Tracking in Low Frame Rate Video: A Cascade Particle Filter with Discriminative Observers of Different Life Spans," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1728-1740, Oct. 2008.
[29] Y. Li, C. Huang, and H. Ai, "Tsinghua Face Detection and Tracking for Clear 2007 Evaluation," Proc. Int'l Workhop Classification of Events, Activities and Relationships, 2007.
[30] Y. Li, C. Huang, and R. Nevatia, "Learning to Associate: Hybridboosted Multi-Target Tracker for Crowded Scene," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[31] K. Mardia, Statistics of Directional Data. Academic Press, 1972.
[32] S.J. McKenna and H. Nait-Charif, "Tracking Human Motion Using Auxiliary Particle Filters and Iterated Likelihood Weighting," Image and Vision Computing, vol. 25, no. 6, pp. 852-862, 2007.
[33] K. Okuma, A. Taleghani, N. De Freitas, J. Little, and D. Lowe, "A Boosted Particle Filter: Multitarget Detection and Tracking," Proc. European Conf. Computer Vision, 2004.
[34] PASCAL VOC, http://pascallin.ecs.soton.ac.uk/challenges VOC/, 2009.
[35] S. Pellegrini, A. Ess, K. Schindler, and L. Van Gool, "You'll Never Walk Alone: Modeling Social Behavior for Multi-Target Tracking," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[36] A. Perera, C. Srinivas, A. Hoogs, G. Brooksby, and W. Hu, "Multi-Object Tracking through Simultaneous Long Occlusions and Split-Merge Conditions," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[37] V. Prisacariu and I. Reid, "Fasthog—A Real-Time GPU Implementation of HOG," Technical Report 2310/09, Dept. of Eng. Science, Oxford Univ., 2009.
[38] C. Rasmussen and G.D. Hager, "Probabilistic Data Association Methods for Tracking Complex Visual Objects," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 560-576, June 2001.
[39] D. Reid, "An Algorithm for Tracking Multiple Targets," IEEE Trans. Automatic Control, vol. 24, no. 6, pp. 843-854, Dec. 1979.
[40] X. Ren, "Finding People in Archive Films through Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[41] D. Schulz, W. Burgard, D. Fox, and A. Cremers, "Tracking Multiple Moving Targets with a Mobile Robot Using Particle Filters and Statistical Data Association," Proc. IEEE Int'l Conf. Robotics and Automation, 2001.
[42] X. Song, J. Cui, H. Zha, and H. Zhao, "Vision-Based Multiple Interacting Targets Tracking via On-Line Supervised Learning," Proc. European Conf. Computer Vision, 2008.
[43] J. Vermaak, A. Doucet, and P. Perez, "Maintaining Multi-Modality through Mixture Tracking," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[44] C. Wojek, G. Dorkó, A. Schulz, and B. Schiele, "Sliding-Windows for Rapid Object Class Localization: A Parallel Technique," Proc. German Assoc. for Pattern Recognition Ann. Pattern Recognition Symp., 2008.
[45] B. Wu and R. Nevatia, "Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet Based Part Detectors," Int'l J. Computer Vision, vol. 75, no. 2, pp. 247-266, 2007.
[46] B. Wu, L. Zhang, V.K. Singh, and R. Nevatia, "Robust Object Tracking Based on Detection with Soft Decision," Proc. IEEE Workshop Motion and Video Computing, 2008.
[47] J. Yang, Z. Shi, P. Vela, and J. Teizer, "Probabilistic Multiple People Tracking through Complex Situations," Proc. IEEE Workshop Performance Evaluation of Tracking and Surveillance, 2009.
[48] L. Zhang, Y. Li, and R. Nevatia, "Global Data Association for Multi-Object Tracking Using Network Flows," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[49] T. Zhao and R. Nevatia, "Tracking Multiple Humans in Complex Situations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1208-1221, Sept. 2004.
60 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool