The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan. (2014 vol.36)
pp: 58-72
Anton Milan , Dept. of Comput. Sci., Tech. Univ. Darmstadt, Darmstadt, Germany
Stefan Roth , Dept. of Comput. Sci., Tech. Univ. Darmstadt, Darmstadt, Germany
Konrad Schindler , Photogrammetry & Remote SensingGroup, ETH Zurich, Zurich, Switzerland
ABSTRACT
Many recent advances in multiple target tracking aim at finding a (nearly) optimal set of trajectories within a temporal window. To handle the large space of possible trajectory hypotheses, it is typically reduced to a finite set by some form of data-driven or regular discretization. In this work, we propose an alternative formulation of multitarget tracking as minimization of a continuous energy. Contrary to recent approaches, we focus on designing an energy that corresponds to a more complete representation of the problem, rather than one that is amenable to global optimization. Besides the image evidence, the energy function takes into account physical constraints, such as target dynamics, mutual exclusion, and track persistence. In addition, partial image evidence is handled with explicit occlusion reasoning, and different targets are disambiguated with an appearance model. To nevertheless find strong local minima of the proposed nonconvex energy, we construct a suitable optimization scheme that alternates between continuous conjugate gradient descent and discrete transdimensional jump moves. These moves, which are executed such that they always reduce the energy, allow the search to escape weak minima and explore a much larger portion of the search space of varying dimensionality. We demonstrate the validity of our approach with an extensive quantitative evaluation on several public data sets.
INDEX TERMS
continuous optimization, Multiobject tracking, tracking-by-detection, visual surveillance,
CITATION
Anton Milan, Stefan Roth, Konrad Schindler, "Continuous Energy Minimization for Multitarget Tracking", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.36, no. 1, pp. 58-72, Jan. 2014, doi:10.1109/TPAMI.2013.103
REFERENCES
[1] M. Andriluka, S. Roth, and B. Schiele, "Monocular 3D Pose Estimation and Tracking by Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[2] A. Andriyenko, S. Roth, and K. Schindler, "An Analytical Formulation of Global Occlusion Reasoning for Multi-Target Tracking," Proc. 11th IEEE Int'l Workshop Visual Surveillance, 2011.
[3] A. Andriyenko and K. Schindler, "Globally Optimal Multi-Target Tracking on a Hexagonal Lattice," Proc. 11th European Conf. Computer Vision, vol. 1, pp. 466-479, 2010.
[4] A. Andriyenko and K. Schindler, "Multi-Target Tracking by Continuous Energy Minimization," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[5] B. Benfold and I. Reid, "Stable Multi-Target Tracking in Real-Time Surveillance Video," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[6] J. Berclaz, F. Fleuret, and P. Fua, "Robust People Tracking with Global Trajectory Optimization," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[7] J. Berclaz, F. Fleuret, and P. Fua, "Multiple Object Tracking Using Flow Linear Programming," Proc. 12th IEEE Int'l Workshop Performance Evaluation of Tracking and Surveillance, Dec. 2009.
[8] J. Berclaz, F. Fleuret, E. Türetken, and P. Fua, "Multiple Object Tracking Using K-Shortest Paths Optimization," IEEE Trans Pattern Analysis and Machine Intelligence, vol. 33, no. 9, pp. 1806-1819, Sept. 2011.
[9] J. Black, T. Ellis, and P. Rosin, "Multi View Image Surveillance and Tracking," Proc. Workshop Motion and Video Computing, Dec. 2002.
[10] M.D. Breitenstein, F. Reichlin, B. Leibe, E. Koller-Meier, and L. Van Gool, "Robust Tracking-by-Detection Using a Detector Confidence Particle Filter," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[11] W. Brendel, M. Amer, and S. Todorovic, "Multiobject Tracking as Maximum Weight Independent Set," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[12] M. Brookes, The Matrix Reference Manual, 2005.
[13] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 886-893, 2005.
[14] M. Eichner and V. Ferrari, "We Are Family: Joint Pose Estimation of Multiple Persons," Proc. 11th European Conf. Computer Vision, pp. 228-242, 2010.
[15] M. Enzweiler, A. Eigenstetter, B. Schiele, and D.M. Gavrila, "Multi-Cue Pedestrian Classification with Partial Occlusion Handling," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[16] A. Ess, B. Leibe, K. Schindler, and L. van Gool, "Robust Multiperson Tracking from a Mobile Platform," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 10, pp. 1831-1846, Oct. 2009.
[17] J. Ferryman and A. Ellis, "PETS2010: Data Set and Challenge," Proc. Seventh IEEE Int'l Conf. Advanced Video and Signal Based Surveillance, 2010.
[18] J.M. Ferryman and A. Shahrokni, "PETS2009: Data Set and Challenge," Proc. 12th IEEE Int'l Workshop Performance Evaluation of Tracking and Surveillance, Dec. 2009.
[19] T.E. Fortmann, Y. Bar-Shalom, and M. Scheffe, "Multi-Target Tracking Using Joint Probabilistic Data Association," Proc. IEEE Conf. Decision and Control, vol. 19, pp. 807-812, Dec. 1980.
[20] W. Ge and R.T. Collins, "Multi-Target Data Association by Tracklets with Unsupervised Parameter Estimation," Proc. British Machine Vision Conf., 2008.
[21] H. Grabner, J. Matas, L. Van Gool, and P.C. Cattin, "Tracking the Invisible: Learning Where the Object Might Be," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[22] P. Green, "Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination," Biometrika, vol. 82, no. 4, pp. 711-732, 1995.
[23] J. Henriques, R. Caseiro, and J. Batista, "Globally Optimal Solution to Multiobject Tracking with Merged Measurements," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[24] H. Jiang, S. Fels, and J.J. Little, "A Linear Programming Approach for Multiple Object Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[25] L. Kratz and K. Nishino, "Tracking with Local Spatio-Temporal Motion Patterns in Extremely Crowded Scenes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[26] C.-H. Kuo, C. Huang, and R. Nevatia, "Multi-Target Tracking by On-Line Learned Discriminative Appearance Models," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[27] B. Leibe, K. Schindler, and L. van Gool, "Coupled Detection and Trajectory Estimation for Multiobject Tracking," Proc. 11th IEEE Int'l Conf. Computer Vision, 2007.
[28] Y. Li, C. Huang, and R. Nevatia, "Learning to Associate: Hybridboosted Multi-Target Tracker for Crowded Scene," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[29] S. Oh, S. Russell, and S. Sastry, "Markov Chain Monte Carlo Data Association for Multi-Target Tracking," IEEE Trans. Automatic Control, vol. 54, no. 3, pp. 481-497, Mar. 2009.
[30] K. Okuma, A. Taleghani, O.D. Freitas, J.J. Little, and D.G. Lowe, "A Boosted Particle Filter: Multitarget Detection and Tracking," Proc. European Conf. Computer Vision, vol. 1, pp. 28-39, 2004.
[31] H. Pirsiavash, D. Ramanan, and C.C. Fowlkes, "Globally-Optimal Greedy Algorithms for Tracking a Variable Number of Objects," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[32] D. Reid, "An Algorithm for Tracking Multiple Targets," IEEE Trans. Automatic Control, vol. 24, no. 6, pp. 843-854, Dec. 1979.
[33] M. Rodriguez, I. Laptev, J. Sivic, and J.-Y. Audibert, "Density-Aware Person Detection and Tracking in Crowds," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[34] H.B. Shitrit, J. Berclaz, F. Fleuret, and P. Fua, "Tracking Multiple People under Global Appearance Constraints," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[35] L. Sigal and M.J. Black, "Measure Locally, Reason Globally: Occlusion-Sensitive Articulated Pose Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[36] R. Stiefelhagen, K. Bernardin, R. Bowers, J.S. Garofolo, D. Mostefa, and P. Soundararajan, "The CLEAR 2006 Evaluation," Proc. First Int'l Evaluation Conf. Classification of Events, Activities and Relationships, 2006.
[37] J. Vermaak, A. Doucet, and P. Pérez, "Maintaining Multi-Modality through Mixture Tracking," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[38] S. Walk, N. Majer, K. Schindler, and B. Schiele, "New Features and Insights for Pedestrian Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[39] C. Wojek, S. Walk, S. Roth, and B. Schiele, "Monocular 3D Scene Understanding with Explicit Occlusion Reasoning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[40] B. Wu and R. Nevatia, "Detection of Multiple, Partially Occluded Humans in a Single Image by Bayesian Combination of Edgelet Part Detectors," Proc. 10th IEEE Int'l Conf. Computer Vision, 2005.
[41] Z. Wu, T.H. Kunz, and M. Betke, "Efficient Track Linking Methods for Track Graphs Using Network-Flow and Set-Cover Techniques," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[42] J. Xing, H. Ai, and S. Lao, "Multiobject Tracking through Occlusions by Local Tracklets Filtering and Global Tracklets Association with Detection Responses," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[43] L. Zhang, Y. Li, and R. Nevatia, "Global Data Association for Multi-object Tracking Using Network Flows," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool