The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.09 - September (2011 vol.33)
pp: 1806-1819
Jérôme Berclaz , École Polytechnique Fédérale de Lausanne, Lausanne
François Fleuret , Idiap Research Institute, Martigny
Engin Türetken , École Polytechnique Fédérale de Lausanne, Lausanne
Pascal Fua , École Polytechnique Fédérale de Lausanne, Lausanne
ABSTRACT
Multi-object tracking can be achieved by detecting objects in individual frames and then linking detections across frames. Such an approach can be made very robust to the occasional detection failure: If an object is not detected in a frame but is in previous and following ones, a correct trajectory will nevertheless be produced. By contrast, a false-positive detection in a few frames will be ignored. However, when dealing with a multiple target problem, the linking step results in a difficult optimization problem in the space of all possible families of trajectories. This is usually dealt with by sampling or greedy search based on variants of Dynamic Programming which can easily miss the global optimum. In this paper, we show that reformulating that step as a constrained flow optimization results in a convex problem. We take advantage of its particular structure to solve it using the k-shortest paths algorithm, which is very fast. This new approach is far simpler formally and algorithmically than existing techniques and lets us demonstrate excellent performance in two very different contexts.
INDEX TERMS
Data association, multiobject tracking, K-shortest paths, linear programming.
CITATION
Jérôme Berclaz, François Fleuret, Engin Türetken, Pascal Fua, "Multiple Object Tracking Using K-Shortest Paths Optimization", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 9, pp. 1806-1819, September 2011, doi:10.1109/TPAMI.2011.21
REFERENCES
[1] J. Giebel, D. Gavrila, and C. Schnorr, "A Bayesian Framework for Multi-Cue 3D Object Tracking," Proc. European Conf. Computer Vision, 2004.
[2] A. Perera, C. Srinivas, A. Hoogs, G. Brooksby, and H. Wensheng, "Multi-Object Tracking through Simultaneous Long Occlusions and Split-Merge Conditions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 666-673, June 2006.
[3] F. Fleuret, J. Berclaz, R. Lengagne, and P. Fua, "Multi-Camera People Tracking with a Probabilistic Occupancy Map," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 267-282, Feb. 2008.
[4] H. Jiang, S. Fels, and J. Little, "A Linear Programming Approach for Multiple Object Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 744-750, 2007.
[5] G.B. Dantzig, Linear Programming and Extensions. Princeton Univ. Press, 1963.
[6] J.W. Suurballe, "Disjoint Paths in a Network," Networks, vol. 4, pp. 125-145, 1974.
[7] J. Black, T. Ellis, and P. Rosin, "Multi-View Image Surveillance and Tracking," Proc. IEEE Workshop Motion and Video Computing, 2002.
[8] A. Mittal and L. Davis, "M2tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene," Computer Vision and Image Understanding, vol. 51, no. 3, pp. 189-203, 2003.
[9] S. Iwase and H. Saito, "Parallel Tracking of All Soccer Players by Integrating Detected Positions in Multiple View Images," Proc. Int'l Conf. Pattern Recognition, pp. 751-754, Aug. 2004.
[10] M. Xu, J. Orwell, and G. Jones, "Tracking Football Players with Multiple Cameras," Proc. Int'l Conf. Image Processing, pp. 2909-2912, Oct. 2004.
[11] D.R. Magee, "Tracking Multiple Vehicles Using Foreground, Background and Motion Models," Image and Vision Computing, vol. 22, no. 2, pp. 143-155, Feb. 2004.
[12] B. Wu and R. Nevatia, "Tracking of Multiple, Partially Occluded Humans Based on Static Body Part Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 951-958, June 2006.
[13] J. Vermaak, A. Doucet, and P. Perez, "Maintaining Multimodality through Mixture Tracking," Proc. IEEE Int'l Conf. Computer Vision, pp. 1110-1116, Oct. 2003.
[14] K. Smith, D. Gatica-Perez, and J.M. Odobez, "Using Particles to Track Varying Numbers of Interacting People," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[15] Z. Khan, T. Balch, and F. Dellaert, "MCMC-Based Particle Filtering for Tracking a Variable Number of Interacting Targets," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1805-1918, Nov. 2005.
[16] C. Yang, R. Duraiswami, and L. Davis, "Fast Multiple Object Tracking via a Hierarchical Particle Filter," Proc. IEEE Int'l Conf. Computer Vision, 2005.
[17] T. Mauthner, M. Donoser, and H. Bischof, "Robust Tracking of Spatial Related Components," Proc. Int'l Conf. Pattern Recognition, 2008.
[18] K. Okuma, A. Taleghani, N. de Freitas, J. Little, and D. Lowe, "A Boosted Particle Filter: Multitarget Detection and Tracking," Proc. European Conf. Computer Vision, May 2004.
[19] W. Du and J. Piater, "Multi-Camera People Tracking by Collaborative Particle Filters and Principal Axis-Based Integration," Proc. Asian Conf. Computer Vision, pp. 365-374, 2007.
[20] Q. Yu, G. Medioni, and I. Cohen, "Multiple Target Tracking Using Spatio-Temporal Markov Chain Monte Carlo Data Association," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[21] E. Maggio, M. Taj, and A. Cavallaro, "Efficient Multi-Target Visual Tracking Using Random Finite Sets," IEEE Trans. Circuits and Systems for Video Technology, vol. 18, no. 8, pp. 1016-1027, Aug. 2008.
[22] C. Huang, B. Wu, and R. Nevatia, "Robust Object Tracking by Hierarchical Association of Detection Responses," Proc. European Conf. Computer Vision, pp. 788-801, 2008.
[23] Y. Li, C. Huang, and R. Nevatia, "Learning to Associate: Hybridboosted Multi-Target Tracker for Crowded Scene," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2009.
[24] C. Beleznai, B. Frühstück, and H. Bischof, "Multiple Object Tracking Using Local PCA," Proc. Int'l Conf. Image Processing, 2006.
[25] W. Ge and R.T. Collins, "Multi-Target Data Association by Tracklets with Unsupervised Parameter Estimation," Proc. British Machine Vision Conf., Sept. 2008.
[26] R. Eshel and Y. Moses, "Homography Based Multiple Camera Detection and Tracking of People in a Dense Crowd," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[27] G.J. Brostow and R. Cipolla, "Unsupervised Bayesian Detection of Independent Motion in Crowds," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 594-601, 2006.
[28] P. Nillius, J. Sullivan, and S. Carlsson, "Multi-Target Tracking— Linking Identities Using Bayesian Network Inference," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2187-2194, 2006.
[29] S. Khan and M. Shah, "Tracking Multiple Occluding People by Localizing on Multiple Scene Planes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 3, pp. 505-519, Mar. 2009.
[30] B. Leibe, K. Schindler, and L.V. Gool, "Coupled Detection and Trajectory Estimation for Multi-Object Tracking," Proc. Int'l Conf. Computer Vision, Oct. 2007.
[31] R.E. Bellman, Dynamic Programming. Princeton Univ. Press, 1957.
[32] J. Wolf, A. Viterbi, and G. Dixon, "Finding the Best Set of K Paths through a Trellis with Application to Multitarget Tracking," IEEE Trans. Aerospace and Electronic Systems, vol. 25, no. 2, pp. 287-296, Mar. 1989.
[33] K. Shafique and M. Shah, "A Noniterative Greedy Algorithm for Multiframe Point Correspondence," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 1, pp. 51-65, Jan. 2005.
[34] P.P.A. Storms and F.C.R. Spieksma, "An LP-Based Algorithm for the Data Association Problem in Multitarget Tracking," Computers and Operations Research, vol. 30, no. 7, pp. 1067-1085, June 2003.
[35] L. Zhang, Y. Li, and R. Nevatia, "Global Data Association for Multi-Object Tracking Using Network Flows," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[36] N. Karmarkar, "A New Polynomial Time Algorithm for Linear Programming," Combinatorica, vol. 4, pp. 373-395, 1984.
[37] A. Ellis, A. Shahrokni, and J. Ferryman, "PETS 2009 and Winter-PETS 2009 Results: A Combined Evaluation," Proc. 12th IEEE Int'l Workshop Performance Evaluation of Tracking and Surveillance, Dec. 2009.
[38] R. Kasturi, D. Goldgof, P. Soundararajan, V. Manohar, J. Garofolo, M. Boonstra, V. Korzhova, and J. Zhang, "Framework for Performance Evaluation of Face, Text, and Vehicle Detection and Tracking in Video: Data, Metrics, and Protocol," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 319-336, Feb. 2009.
[39] Proc. 11th IEEE Int'l Workshop Performance Evaluation of Tracking and Surveillance, http:/pets2009.net, 2011.
[40] J. Berclaz, F. Fleuret, and P. Fua, "POM: Probabilistic Occupancy Map," http://cvlab.epfl.ch/software/pomindex.php , 2007.
[41] K. Bernardin and R. Stiefelhagen, "Evaluating Multiple Object Tracking Performance: The Clear MOT Metrics," EURASIP J. Image and Video Processing, vol. 2008, pp. 1-10, 2008.
[42] A. Makhorin, "GLPK—GNU Linear Programming Kit," http://www.gnu.org/softwareglpk/, 2008.
[43] A. Ghouila-Houri, "Caractérisation Des Matrices Totalement Unimodulaires," Comptes Rendus Hebdomadaires des Séances de l'Académie des Sciences, vol. 254, pp. 1192-1194, 1962.
[44] A.J. Hoffman and J.B. Kruskal, "Integral Boundary Points of Convex Polyhedra," Linear Inequalities and Related Systems, pp. 223-246, Princeton Univ. Press, 1956.
[45] E.W. Dijkstra, "A Note on Two Problems in Connexion with Graphs," Numerische Mathematik, vol. 1, pp. 269-271, 1959.
[46] T.H. Cormen, C.E. Leiserson, R.L. Rivest, and C. Stein, Introduction to Algorithms, second ed. MIT Press, 2001.
[47] Y. Ma, Q. Yu, and I. Cohen, "Target Tracking with Incomplete Detection," Computer Vision and Image Understanding, vol. 113, no. 4, pp. 580-587, 2009.
25 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool