The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - October (2009 vol.31)
pp: 1831-1846
Andreas Ess , ETH Zurich, Switzerland
Bastian Leibe , RWTH Aachen University, Germany
Konrad Schindler , TU Darmstadt, Germany
Luc van Gool , ETH Zurich, Switzerland and KU Leuven, Belgium
ABSTRACT
In this paper, we address the problem of multiperson tracking in busy pedestrian zones using a stereo rig mounted on a mobile platform. The complexity of the problem calls for an integrated solution that extracts as much visual information as possible and combines it through cognitive feedback cycles. We propose such an approach, which jointly estimates camera position, stereo depth, object detection, and tracking. The interplay between those components is represented by a graphical model. Since the model has to incorporate object-object interactions and temporal links to past frames, direct inference is intractable. We, therefore, propose a two-stage procedure: for each frame, we first solve a simplified version of the model (disregarding interactions and temporal continuity) to estimate the scene geometry and an overcomplete set of object detections. Conditioned on these results, we then address object interactions, tracking, and prediction in a second step. The approach is experimentally evaluated on several long and difficult video sequences from busy inner-city locations. Our results show that the proposed integration makes it possible to deliver robust tracking performance in scenes of realistic complexity.
INDEX TERMS
Mobile vision, multiobject tracking, pedestrian detection, stereo depth, visual odometry, graphical model.
CITATION
Andreas Ess, Bastian Leibe, Konrad Schindler, Luc van Gool, "Robust Multiperson Tracking from a Mobile Platform", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 10, pp. 1831-1846, October 2009, doi:10.1109/TPAMI.2009.109
REFERENCES
[1] M. Andriluka, S. Roth, and B. Schiele, “People-Tracking-by-Detection and People-Detection-by-Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[2] S. Avidan, “Ensemble Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[3] H. Bay, A. Ess, T. Tuytelaars, and L. van Gool, “Speeded-Up Robust Features (SURF),” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346-359, 2008.
[4] J. Berclaz, F. Fleuret, and P. Fua, “Robust People Tracking with Global Trajectory Optimization,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[5] C. Bibby and I. Reid, “Simultaneous Localisation and Mapping in Dynamic Environments (SLAMIDE) with Reversible Data Association,” Proc. Conf. Robotics Science and Systems, 2007.
[6] C.M. Bishop, Pattern Recognition and Machine Learning. Springer Verlag, 2006.
[7] E. Borenstein and S. Ullman, “Class-Specific, Top-Down Segmentation,” Proc. European Conf. Computer Vision, 2002.
[8] D. Comaniciu, V. Ramesh, and P. Meer, “Kernel-Based Object Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 5, pp. 564-575, May 2003.
[9] I.J. Cox, “A Review of Statistical Data Association Techniques for Motion Correspondence,” Int'l J. Computer Vision, vol. 10, no. 1, pp. 53-66, 1993.
[10] N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[11] N. Dalal, B. Triggs, and C. Schmid, “Human Detection Using Oriented Histograms of Flow and Appearance,” Proc. European Conf. Computer Vision, 2006.
[12] A.J. Davison, “Real-Time Simultaneous Localization and Mapping with a Single Camera,” Proc. Int'l Conf. Computer Vision, 2003.
[13] E. Eade and T. Drummond, “Scalable Monocular Slam,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[14] A. Ess, B. Leibe, K. Schindler, and L. van Gool, “A Mobile Vision System for Robust Multi-Person Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[15] A. Ess, B. Leibe, and L. van Gool, “Depth and Appearance for Mobile Scene Analysis,” Proc. Int'l Conf. Computer Vision, 2007.
[16] P. Felzenszwalb, D. McAllester, and D. Ramanan, “A Discriminatively Trained, Multiscale, Deformable Part Model,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[17] P.F. Felzenszwalb and D.P. Huttenlocher, “Efficient Belief Propagation for Early Vision,” Int'l J. Computer Vision, vol. 70, pp. 41-54, , 2006.
[18] T.E. Fortmann, Y. Bar Shalom, and M. Scheffe, “Sonar Tracking of Multiple Targets Using Joint Probabilistic Data Association,” IEEE J. Oceanic Eng., vol. 8, no. 3, pp. 173-184, July 1983.
[19] D.M. Gavrila and S. Munder, “Multi-Cue Pedestrian Detection and Tracking from a Moving Vehicle,” Int'l J. Computer Vision, vol. 73, pp. 41-59, 2007.
[20] A. Gelb, Applied Optimal Estimation. MIT Press, 1996.
[21] H. Grabner and H. Bischof, “On-Line Boosting and Vision,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[22] D. Hähnel, R. Triebel, W. Burgard, and S. Thrun, “Map Building with Mobile Robots in Dynamic Environments,” Proc. Int'l Conf. Robotics and Automation, 2003.
[23] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004.
[24] D. Hoiem, A.A. Efros, and M. Hebert, “Putting Objects in Perspective,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[25] M. Isard and A. Blake, “CONDENSATION—Conditional Density Propagation for Visual Tracking,” Int'l J. Computer Vision, vol. 29, no. 1, pp. 5-28, 1998.
[26] R. Kaucic, A.G. Perera, G. Brooksby, J. Kaufhold, and A. Hoogs, “A Unified Framework for Tracking through Occlusions and across Sensor Gaps,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[27] R. Labayrade, D. Aubert, and J.-P. Tarel, “Real Time Obstacle Detection on Non Flat Road Geometry through ‘v-Disparity’ Representation,” Proc. IEEE Intelligent Vehicle Symp., 2002.
[28] O. Lanz, “Approximate Bayesian Multibody Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp.1436-1449, Sept. 2006.
[29] B. Leibe, N. Cornelis, K. Cornelis, and L. van Gool, “Dynamic 3D Scene Analysis from a Moving Vehicle,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[30] B. Leibe, A. Leonardis, and B. Schiele, “Robust Object Detection with Interleaved Categorization and Segmentation,” Int'l J. Computer Vision, vol. 77, nos. 1-3, pp. 259-289, 2008.
[31] B. Leibe, K. Schindler, N. Cornelis, and L. van Gool, “Coupled Detection and Tracking from Static Cameras and Moving Vehicles,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1683-1698, Oct. 2008.
[32] B. Leibe, K. Schindler, and L. van Gool, “Coupled Detection and Trajectory Estimation for Multi-Object Tracking,” Proc. Int'l Conf. Computer Vision, 2007.
[33] B. Leibe, E. Seemann, and B. Schiele, “Pedestrian Detection in Crowded Scenes,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[34] T. Li, V. Khallem, D. Singaraju, and R. Vidal, “Projective Factorization of Multiple Rigid-Body Motions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[35] A. Makadia, C. Geyer, S. Sastry, and K. Daniilidis, “Radon-Based Structure from Motion without Correspondences,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[36] K. Murphy, A. Torralba, and W.T. Freeman, “Using the Forest to See the Trees: A Graphical Model Relating Features, Objects, and Scenes,” Proc. Conf. Neural Information Processing Systems, 2003.
[37] D. Nistér, “A Minimal Solution to the Generalised 3-Point Pose Problem,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[38] D. Nistér, O. Naroditsky, and J.R. Bergen, “Visual Odometry,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[39] K. Okuma, A. Taleghani, N. de Freitas, J. Little, and D. Lowe, “A Boosted Particle Filter: Multitarget Detection and Tracking,” Proc. European Conf. Computer Vision, 2004.
[40] B. Ommer and J.M. Buhmann, “Object Categorization by Compositional Graphical Models,” Proc. Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, 2005.
[41] K.E. Ozden, K. Schindler, and L. van Gool, “Simultaneous Segmentation and 3D Reconstruction of Monocular Image Sequences,” Proc. Int'l Conf. Computer Vision, 2007.
[42] J. Pearl, Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, Inc., 1988.
[43] D.B. Reid, “An Algorithm for Tracking Multiple Targets,” IEEE Trans. Automatic Control, vol. 24, no. 6, pp. 843-854, Dec. 1979.
[44] C. Rother, V. Kolmogorov, V.S. Lempitsky, and M. Szummer, “Optimizing Binary mrfs via Extended Roof Duality,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[45] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int'l J. Computer Vision, vol. 47, pp. 7-42, 2002.
[46] K. Schindler, U. James, and H. Wang, “Perspective n-View Multibody Structure-and-Motion through Model Selection,” Proc. European Conf. Computer Vision, 2006.
[47] S. Se, D. Lowe, and J. Little, “Global Localization Using Distinctive Visual Features,” Proc. Int'l Conf. Intelligent Robots and Systems, 2002.
[48] V. Sharma and J. Davis, “Integrating Appearance and Motion Cues for Simultaneous Detection and Segmentation of Pedestrians,” Proc. Int'l Conf. Computer Vision, 2007.
[49] C. Stauffer and W.E.L. Grimson, “Adaptive Background Mixture Models for Real-Time Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[50] E.B. Sudderth, A.B. Torralba, W.T. Freeman, and A.S. Willsky, “Learning Hierarchical Models of Scenes, Objects, and Parts,” Proc. Int'l Conf. Computer Vision, 2005.
[51] A. Torralba, “Contextual Priming for Object Detection,” Int'l J. Computer Vision, vol. 53, no. 2, pp. 153-167, 2003.
[52] K. Toyama, J. Krumm, B. Brumitt, and B. Meyers, “Wallflower: Principles and Practice of Background Maintenance,” Proc. Int'l Conf. Computer Vision, 1999.
[53] O. Tuzel, F. Porikli, and P. Meer, “Human Detection via Classification on Riemannian Manifolds,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[54] P. Viola, M. Jones, and D. Snow, “Detecting Pedestrians Using Patterns of Motion and Appearance,” Proc. Int'l Conf. Computer Vision, 2003.
[55] C.C. Wang, C. Thorpe, S. Thrun, M. Hebert, and H. Durrant, “Simultaneous Localization, Mapping and Moving Object Tracking,” Int'l J. Robotics Research, vol. 26, pp. 889-916, 2007.
[56] C. Wojek, G. Dorkó, A. Schulz, and B. Schiele, “Sliding-Windows for Rapid Object Class Localization: A Parallel Technique,” Proc. Symp. German Assoc. for Pattern Recognition (DAGM), 2008.
[57] B. Wu and R. Nevatia, “Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet Part Detectors,” Int'l J. Computer Vision, vol. 75, no. 2, pp.247-266, 2007.
[58] B. Wu and R. Nevatia, “Simultaneous Object Detection and Segmentation by Boosting Local Shape Feature Based Classifier,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[59] F. Yan, A. Kostin, W.J. Christmas, and J. Kittler, “A Novel Data Association Algorithm for Object Tracking in Clutter with Application to Tennis Video Analysis,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[60] L. Zhang, Y. Li, and R. Nevatia, “Global Data Association for Multi-Object Tracking Using Network Flows,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[61] Z. Zhu, T. Oskiper, O. Naroditsky, S. Samarasekera, H.S. Sawhney, and R. Kumar, “An Improved Stereo-Based Visual Odometry System,” Proc. Workshop Performance Metrics for Intelligent Systems, 2006.
50 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool