Subscribe
Issue No.10 - October (2008 vol.30)
pp: 1683-1698
Bastian Leibe , ETH Zurich, Zurich
Konrad Schindler , ETH Zurich, Zurich
Nico Cornelis , KU Leuven, Leuven
Luc Van Gool , ETH Zurich KU Leuven, Zurich Leuven
ABSTRACT
We present a novel approach for multi-object tracking which considers object detection and spacetime trajectory estimation as a coupled optimization problem. Our approach is formulated in an MDL hypothesis selection framework, which allows it to recover from mismatches and temporarily lost tracks. Building upon a multi-view/multi-category object detector, it localizes cars and pedestrians in the input images. The 2D object detections are converted to 3D observations, which are accumulated in a world coordinate frame. Trajectory analysis in a spacetime window yields physically plausible trajectory candidates. Tracking is achieved by performing model selection after every frame. At each time instant, our approach searches for the globally optimal set of spacetime trajectories which provides the best explanation for the current image and all evidence collected so far, while satisfying the constraints that no two objects may occupy the same physical space, nor explain the same image pixels at any time. Successful trajectory hypotheses are then fed back to guide object detection in future frames. The resulting approach can initialize automatically and track a large and varying number of objects from both static and moving cameras. We evaluate our approach on several challenging video sequences with both a surveillance-type scenario and a scenario where the input videos are taken from a moving vehicle.
INDEX TERMS
Vision and Scene Understanding, Video analysis, Scene Analysis, Object recognition, Tracking
CITATION
Bastian Leibe, Konrad Schindler, Nico Cornelis, Luc Van Gool, "Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 10, pp. 1683-1698, October 2008, doi:10.1109/TPAMI.2008.170
REFERENCES
 [1] S. Avidan, “Ensemble Tracking,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2005. [2] J. Berclaz, F. Fleuret, and P. Fua, “Robust People Tracking with Global Trajectory Optimization,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 744-750, 2006. [3] M. Betke, E. Haritaoglu, and L. Davis, “Real-Time Multiple Vehicle Tracking from a Moving Vehicle,” Machine Vision and Applications, vol. 12, no. 2, pp. 69-83, 2000. [4] E. Boros and P. Hammer, “Pseudo-Boolean Optimization,” Discrete Applied Math., vol. 123, nos. 1-3, pp. 155-225, 2002. [5] D. Comaniciu and P. Meer, “Mean Shift: A Robust Approach toward Feature Space Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002. [6] D. Comaniciu, V. Ramesh, and P. Meer, “Kernel-Based Object Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 5, pp. 564-575, May 2003. [7] N. Cornelis, K. Cornelis, and L. Van Gool, “Fast Compact City Modeling for Navigation Pre-Visualization,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006. [8] N. Cornelis, B. Leibe, K. Cornelis, and L. Van Gool, “3D Urban Scene Modeling Integrating Recognition and Reconstruction,” Int'l J. Computer Vision, vol. 78, nos. 2-3, pp. 121-141, 2008. [9] I. Cox, “A Review of Statistical Data Association Techniques for Motion Correspondence,” Int'l J. Computer Vision, vol. 10, no. 1, pp. 53-66, 1993. [10] N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition), 2005. [11] A. Ess, B. Leibe, and L. Van Gool, “Depth and Appearance for Mobile Scene Analysis,” Proc. IEEE Int'l Conf. Computer Vision, 2007. [12] M. Everingham et al., “The 2005 PASCAL Visual Object Class Challenge,” Machine Learning Challenges: Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Textual Entailment, LNAI 3944, Springer, 2006. [13] T. Fortmann, Y. Bar Shalom, and M. Scheffe, “Sonar Tracking of Multiple Targets Using Joint Probabilistic Data Association,” IEEE J. Oceanic Eng., vol. 8, no. 3, pp. 173-184, 1983. [14] D. Gavrila and V. Philomin, “Real-Time Object Detection for Smart Vehicles,” Proc. IEEE Int'l Conf. Computer Vision, pp. 87-93, 1999. [15] A. Gelb, Applied Optimal Estimation. MIT Press, 1996. [16] J. Giebel, D. Gavrila, and C. Schnörr, “A Bayesian Framework for Multi-Cue 3D Object Tracking,” Proc. European Conf. Computer Vision, 2004. [17] H. Grabner and H. Bischof, “On-Line Boosting and Vision,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 260-267, 2006. [18] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000. [19] D. Hoiem, A. Efros, and M. Hebert, “Geometric Context from a Single Image,” Proc. IEEE Int'l Conf. Computer Vision, 2005. [20] D. Hoiem, A. Efros, and M. Hebert, “Putting Objects into Perspective,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006. [21] M. Isard and A. Blake, “CONDENSATION-Conditional Density Propagation for Visual Tracking,” Int'l J. Computer Vision, vol. 29, no. 1, 1998. [22] R. Kaucic, A. Perera, G. Brooksby, J. Kaufhold, and A. Hoogs, “A Unified Framework for Tracking through Occlusions and Across Sensor Gaps,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2005. [23] J. Keuchel, “Multiclass Image Labeling with Semidefinite Programming,” Proc. European Conf. Computer Vision, pp. 454-467, 2006. [24] D. Koller, K. Daniilidis, and H.-H. Nagel, “Model-Based Object Tracking in Monocular Image Sequences of Road Traffic Scenes,” Int'l J. Computer Vision, vol. 10, no. 3, pp. 257-281, 1993. [25] M. Kumar, P. Torr, and A. Zisserman, “Solving Markov Random Fields Using Second Order Cone Programming Relaxations,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006. [26] O. Lanz, “Approximate Bayesian Multibody Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp.1436-1449, Sept. 2006. [27] B. Leibe, N. Cornelis, K. Cornelis, and L. Van Gool, “Dynamic 3D Scene Analysis from a Moving Vehicle,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007. [28] B. Leibe, A. Leonardis, and B. Schiele, “Robust Object Detection with Interleaved Categorization and Segmentation,” Int'l J. Computer Vision, vol. 77, no. 1-3, pp. 259-289, 2008. [29] B. Leibe, K. Mikolajczyk, and B. Schiele, “Segmentation Based Multi-Cue Integration for Object Detection,” Proc. British Machine Vision Conf., 2006. [30] B. Leibe, K. Schindler, and L. Van Gool, “Coupled Detection and Trajectory Estimation for Multi-Object Tracking,” Proc. IEEE Int'l Conf. Computer Vision, 2007. [31] B. Leibe, E. Seemann, and B. Schiele, “Pedestrian Detection in Crowded Scenes,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2005. [32] A. Leonardis, A. Gupta, and R. Bajcsy, “Segmentation of Range Images as the Search for Geometric Parametric Models,” Int'l J. Computer Vision, vol. 14, pp. 253-277, 1995. [33] D. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004. [34] K. Mikolajczyk, B. Leibe, and B. Schiele, “Multiple Object Class Detection with a Generative Model,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006. [35] K. Mikolajczyk and C. Schmid, “A Performance Evaluation of Local Descriptors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, Oct. 2005. [36] K. Nummiaro, E. Koller-Meier, and L. Van Gool, “An Adaptive Color-Based Particle Filter,” Image and Vision Computing, vol. 21, no. 1, pp. 99-110, 2003. [37] K. Okuma, A. Taleghani, N. de Freitas, J. Little, and D. Lowe, “A Boosted Particle Filter: Multitarget Detection and Tracking,” Proc. European Conf. Computer Vision, 2004. [38] V. Philomin, R. Duraiswami, and L. Davis, “Pedestrian Tracking from a Moving Vehicle,” Proc. Intelligent Vehicles Symp., pp. 350-355, 2000. [39] D. Reid, “An Algorithm for Tracking Multiple Targets,” IEEE Trans. Automatic Control, vol. 24, no. 6, pp. 843-854, 1979. [40] C. Rother, V. Kolmogorov, V. Lempitsky, and M. Szummer, “Optimizing Binary MRFs via Extended Roof Duality,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007. [41] K. Schindler, J. U, and H. Wang, “Perspective $n\hbox{-}{\rm View}$ Multibody Structure-and-Motion through Model Selection,” Proc. European Conf. Computer Vision, pp. 606-619, 2006. [42] C. Stauffer and W. Grimson, “Adaptive Background Mixture Models for Realtime Tracking,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1999. [43] P. Viola and M. Jones, “Robust Real-Time Face Detection,” Int'l J. Computer Vision, vol. 57, no. 2, pp. 137-154, 2004. [44] P. Viola, M. Jones, and D. Snow, “Detecting Pedestrians Using Patterns of Motion and Appearance,” Proc. IEEE Int'l Conf. Computer Vision, pp. 734-741, 2003. [45] B. Wu and R. Nevatia, “Detection of Multiple, Partially Occluded Humans in a Single Image by Bayesian Combination of Edgelet Part Detectors,” Proc. IEEE Int'l Conf. Computer Vision, 2005. [46] B. Wu and R. Nevatia, “Tracking of Multiple, Partially Occluded Humans Based on Static Body Part Detections,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006. [47] F. Yan, A. Kostin, W. Christmas, and J. Kittler, “A Novel Data Association Algorithm for Object Tracking in Clutter with Application to Tennis Video Analysis,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.
23 ms
(Ver 2.0)

Marketing Automation Platform