This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Tracking Multiple Occluding People by Localizing on Multiple Scene Planes
March 2009 (vol. 31 no. 3)
pp. 505-519
ASCII Text
x 
 
Saad M. Khan, Mubarak Shah, "Tracking Multiple Occluding People by Localizing on Multiple Scene Planes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 3, pp. 505-519, March, 2009.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2008.102,
author = {Saad M. Khan and Mubarak Shah},
title = {Tracking Multiple Occluding People by Localizing on Multiple Scene Planes},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {31},
number = {3},
issn = {0162-8828},
year = {2009},
pages = {505-519},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2008.102},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Tracking Multiple Occluding People by Localizing on Multiple Scene Planes
IS - 3
SN - 0162-8828
SP505
EP519
EPD - 505-519
A1 - Saad M. Khan,
A1 - Mubarak Shah,
PY - 2009
KW - Tracking
KW - Sensor fusion
VL - 31
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Saad M. Khan, University of Central Florida, Orlando
Mubarak Shah, University of Central Florida, Orlando
Occlusion and lack of visibility in crowded and cluttered scenes make it difficult to track individual people correctly and consistently, particularly in a single view. We present a multi-view approach to solving this problem. In our approach we neither detect nor track objects from any single camera or camera pair; rather evidence is gathered from all the cameras into a synergistic framework and detection and tracking results are propagated back to each view. Unlike other multi-view approaches that require fully calibrated views our approach is purely image-based and uses only 2D constructs. To this end we develop a planar homographic occupancy constraint that fuses foreground likelihood information from multiple views, to resolve occlusions and localize people on a reference scene plane. For greater robustness this process is extended to multiple planes parallel to the reference plane in the framework of plane to plane homologies. Our fusion methodology also models scene clutter using the Schmieder and Weathersby clutter measure, which acts as a confidence prior, to assign higher fusion weight to views with lesser clutter. Detection and tracking are performed simultaneously by graph cuts segmentation of tracks in the space-time occupancy likelihood data. Experimental results with detailed qualitative and quantitative analysis, are demonstrated in challenging multi-view, crowded scenes.

[1] A. Azarbayejani and A. Pentland, “Real-Time Self-Calibrating Stereo Person Tracking Using 3D Shape Estimation from Blob Features,” Proc. 13th Int'l Conf. Pattern Recognition, 1996.
[2] J. Berclaz, F. Fleuret, and P. Fua, “Robust People Tracking with Global Trajectory Optimization,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[3] Y. Boukov and M. Jolly, “Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images,” Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001.
[4] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, 2001.
[5] G. Brostow and R. Cipolla, “Unsupervised Bayesian Detection of Independent Motion in Crowds,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[6] Q. Cai and J. Aggarwal, “Automatic Tracking of Human Motion in Indoor Scenes Across Multiple Synchronized Video Streams,” Proc. Sixth IEEE Int'l Conf. Computer Vision, 1998.
[7] T. Chang and S. Gong, “Tracking Multiple People with a Multi-Camera System,” Proc. IEEE Workshop Multi-Object Tracking, 2001.
[8] A. Criminisi, I. Reid, and A. Zisserman, “Single View Metrology,” Int'l J. Computer Vision, 1989.
[9] T. Darrell, D. Demirdjian, N. Checka, and P. Felzenszwalb, “Plan-View Trajectory Estimation with Dense Stereo Background Models,” Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001.
[10] S. Dockstader and A. Tekalp, “Multiple Camera Fusion for Multi-Object Tracking,” Proc. IEEE Workshop Multi-Object Tracking, 2001.
[11] A. Elfes, “Occupancy Grids: A Probabilistic Framework for Robot Perception and Navigation,” PhD dissertation, 1989.
[12] J. Franco and E. Boyer, “Fusion of Multi-View Silhouette Cues Using a Space Occupancy Grid,” Proc. 10th IEEE Int'l Conf. Computer Vision, 2005.
[13] D. Freedman and M.W. Turek, “Illumination-Invariant Tracking via Graph Cuts,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[14] D. Freedman and T. Zhang, “Interactive Graph Cut Based Segmentation with Shape Priors,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[15] J. Gibson, The Ecological Approach to Visual Perception. Houghton Mifflen, 1979.
[16] M. Han, W. Xu, H. Tao, and Y. Gong, “An Algorithm for Multiple Object Trajectory Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[17] I. Haritaoglu, D. Harwood, and L. Davis, “W4: Real-Time Surveillance of People and Their Activities,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, 2000.
[18] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2002.
[19] W. Hu, M. Hu, X. Zhou, T. Tan, J. Lou, and S. Maybank, “Principal Axis-Based Correspondence between Multiple Cameras for People Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, 2006.
[20] Y. Huang and I. Essa, “Tracking Multiple Objects Through Occlusions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[21] M. Isard and J. MacCormick, “Bramble: A Bayesian Multiple-Blob Tracker,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001.
[22] R. Jain and K. Wakimoto, “Multiple Perspective Interactive Video,” Proc. IEEE Int'l Conf. Multimedia Computing and Systems, 1995.
[23] N. Jojic and B. Frey, “Learning Flexible Sprites in Video Layers,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001.
[24] J. Kang, I. Cohen, and G. Medioni, “Continuous Tracking within and across Camera Streams,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[25] P. Kelly, A. Katkere, D. Kuramura, S. Moezzi, S. Chatterjee, and R. Jain, “An Architecture for Multiple Perspective Interactive Video,” Proc. Third ACM Int'l Conf. Multimedia, 1995.
[26] S. Khan and M. Shah, “Consistent Labeling of Tracked Objects in Multiple Cameras with Overlapping Fields of View,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, 2003.
[27] S.M. Khan and M. Shah, “A Multi-View Approach to Tracking People in Crowded Scenes Using a Planar Homography Constraint,” Proc. Ninth European Conf. Computer Vision, 2006.
[28] Z. Khan, T.R. Balch, and F. Dellaert, “An MCMC-Based Particle Filter for Tracking Multiple Interacting Targets,” Proc. Eighth European Conf. Computer Vision, 2004.
[29] K. Kim and L. Davis, “Multi-Camera Tracking and Segmentation of Occluded People on Ground Plane Using Search-Guided Particle Filtering,” Proc. Ninth European Conf. Computer Vision, 2006.
[30] P. Kornprobst and G. Medioni, “Tracking Segmented Objects Using Tensor Voting,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000.
[31] J. Krumm, S. Harris, B. Meyers, B. Brumitt, M. Hale, and S. Shafer, “Multi-Camera Multi-Person Tracking for Easy Living,” Proc. Third IEEE Int'l Workshop Visual Surveillance, 2000.
[32] D. Lowe, “Distinctive Image Features from Scale Invariant Keypoints,” Int'l J. Computer Vision, 2004.
[33] F. Lv, T. Zhao, and R. Nevatia, “Self-Calibration of a Camera from Video of a Walking Human,” Proc. 16th Int'l Conf. Pattern Recognition, 2002.
[34] J. MacCormick and A. Blake, “A Probabilistic Exclusion Principle for Tracking Multiple Objects,” Int'l J. Computer Vision, 2000.
[35] D. Marr, Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. W.H. Freeman, 1982.
[36] S. McKenna, S. Jabri, Z. Duric, A. Rosenfeld, and H. Wechsler, “Tracking Groups of People,” Computer Vision and Image Understanding, 2000.
[37] A. Mittal and S. Larry, “M2tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene,” Int'l J. Computer Vision, 2002.
[38] A. Nakazawa, H. Kato, and S. Inokuchi, “Human Tracking Using Distributed Vision Systems,” Proc. 14th Int'l Conf. Pattern Recognition, 1998.
[39] U. Neisser, Cognition and Reality: Principles and Implications of Cognitive Psychology. W.H. Freeman, 1976.
[40] K. Okuma, A. Taleghani, N. de Freitas, J. Little, and D. Lowe, “A Boosted Particle Filter: Multitarget Detection and Tracking,” Proc. Eighth European Conf. Computer Vision, 2004.
[41] J. Orwell, S. Massey, P. Remagnino, D. Greenhill, and G. Jones, “A Multi-Agent Framework for Visual Surveillance,” Proc. IEEE Int'l Conf. Image Processing, 1999.
[42] S. Park and M.M. Trivedi, “Analysis and Query of Person-Vehicle Interactions in Homography Domain,” Proc. Fourth ACM Int'l Workshop Video Surveillance and Sensor Networks, 2006.
[43] S. Park and M.M. Trivedi, “Multi-Perspective Video Analysis of Persons and Vehicles for Enhanced Situational Awareness,” Proc. IEEE Int'l Conf. Intelligence and Security Informatics, 2006.
[44] A. Perera, C. Srinivas, A. Hoogs, G. Brooksby, and W. Hu, “Multi-Object Tracking Through Simultaneous Long Occlusions and Split and Merge Conditions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[45] A. Poore, “Multidimensional Assignments and Multitarget Tracking,” Proc. DIMACS Workshop, 1995.
[46] D. Reid, “An Algorithm for Tracking Multiple Targets,” IEEE Trans. Automatic Control, 1979.
[47] R. Rosales and S. Sclaroff, “3D Trajectory Recovery for Tracking Multiple Objects and Trajectory Guided Recognition of Actions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[48] C. Rother, “A New Approach for Vanishing Point Detection in Architectural Environments,” Proc. 13th British Machine Vision Conf., 2002.
[49] S. Rotman, G. Tidhar, and M. Kowalczyk, “Clutter Metrics for Target Detection Systems,” IEEE Trans. Aerospace and Electronic Systems, 1994.
[50] K. Sato, T. Maeda, H. Kato, and S. Inokuchi, “Cad-Based Object Tracking with Distributed Monocular Camera for Security Monitoring,” Proc. Second IEEE CAD-Based Vision Workshop, 1994.
[51] D. Schmieder and M. Weathersby, “Performance Detection with Variable Clutter Resolution,” IEEE Trans. Aerospace and Electronic Systems, 1983.
[52] A. Senior, A. Hampapur, Y. Tian, L. Brown, S. Pankanti, and R. Bolle, “Appearance Models for Occlusion Handling,” Proc. Second IEEE Workshop Performance Evaluation of Tracking and Surveillance, 2001.
[53] I. Sethi and R. Jain, “Finding Trajectories of Feature Points in a Monocular Image Sequence,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, 1987.
[54] H. Sidenbladh, M. Black, and D. Fleet, “Stochastic Tracking of 3D Human Figures Using 2D Image Motion,” Proc. Sixth European Conf. Computer Vision (ECCV), 2000.
[55] C. Stauffer and W. Grimson, “Adaptive Background Mixture Models for Real-Time Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[56] H. Tao, H.S. Sawhney, and R. Kumar, “Object Tracking with Bayesian Estimation of Dynamic Layer Representations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, 2002.
[57] S. Thrun, “Learning Occupancy Grid Maps with Forward Sensor Models,” J. Autonomous Robots, 2003.
[58] C. Wren, A. Azarbayejani, T. Darell, and A. Pentland, “Pfinder: Real-Time Tracking of Human Body,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, 1997.
[59] Y. Wu, T. Yu, and G. Hua, “Tracking Appearances with Occlusions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[60] D.B. Yang, H.H. Gonzalez-Banos, and L.J. Guibas, “Counting People in Crowds with a Real-Time Network of Simple Image Sensors,” Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
[61] A. Yilmaz, O. Javed, and M. Shah, “Object Tracking: A Survey,” ACM J. Computing Surveys, 2006.
[62] A. Yilmaz, X. Li, and M. Shah, “Contour-Based Object Tracking with Occlusion Handling in Video Acquired Using Mobile Cameras,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, 2004.
[63] T. Zhao and R. Nevatia, “Tracking Multiple Humans in Complex Situations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, 2004.
[64] T. Zhao and R. Nevatia, “Tracking Multiple Humans in Crowded Environment,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.

Index Terms:
Tracking, Sensor fusion
Citation:
Saad M. Khan, Mubarak Shah, "Tracking Multiple Occluding People by Localizing on Multiple Scene Planes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 3, pp. 505-519, March 2009, doi:10.1109/TPAMI.2008.102
Usage of this product signifies your acceptance of the Terms of Use.