The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2008 vol.30)
pp: 1198-1211
ABSTRACT
Segmentation and tracking of multiple humans in crowded situations is made difficult by interobject occlusion. We propose a model based approach to interpret the image observations by multiple, partially occluded human hypotheses in a Bayesian framework. We define a joint image likelihood for multiple humans based on the appearance of the humans, the visibility of body obtained by occlusion reasoning, and foreground/background separation. The optimal solution is obtained by using an efficient sampling method, data-driven Markov chain Monte Carlo (DDMCMC), which uses image observations for proposal probabilities. Knowledge of various aspects including human shape, camera model, and image cues are integrated in one theoretically sound framework. We present experimental results and quantitative evaluation, demonstrating that the resulting approach is effective for very challenging data.
INDEX TERMS
Multiple Human Segmentation, Multiple Human Tracking, Markov chain Monte Carlo
CITATION
Tao Zhao, Ram Nevatia, Bo Wu, "Segmentation and Tracking of Multiple Humans in Crowded Environments", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 7, pp. 1198-1211, July 2008, doi:10.1109/TPAMI.2007.70770
REFERENCES
[1] G. Borgefors, “Distance Transformations in Digital Images,” Computer Vision, Graphics, and Image Processing, vol. 34, no. 3, pp.344-371, 1986.
[2] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[3] I. Cohen and G. Medioni, “Detecting and Tracking Moving Objects for Video Surveillance,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2319-2326, 1999.
[4] R.T. Collins, “Mean-Shift Blob Tracking through Scale Space,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp.234-240, 2003.
[5] D. Comaniciu and P. Meer, “Mean Shift: A Robust Approach toward Feature Space Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[6] D. Comaniciu and P. Meer, “Kernel-Based Object Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 5, pp.564-577, May 2003.
[7] L. Davis, V. Philomin, and R. Duraiswami, “Tracking Humans from a Moving Platform,” Proc. Int'l Conf. Pattern Recognition, vol. 4, pp. 171-178, 2000.
[8] J. Deutscher, A. Blake, and I. Reid, “Articulated Body Motion Capture by Annealed Particle Filtering,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 126-133, 2000.
[9] A. Elgammal and L. Davis, “Probabilistic Framework for Segmenting People under Occlusion,” Proc. Eighth Int'l Conf. Computer Vision, vol. 2, pp. 145-152, 2001.
[10] A. Elgammal, R. Duraiswami, D. Harwood, and L. Davis, “Background and Foreground Modeling Using Non-Parametric Kernel Density Estimation for Visual Surveillance,” Proc. IEEE, vol. 90, no. 7, pp. 1151-1163, 2002.
[11] F. Fleuret, R. Lengagne, and P. Fua, “Fixed Point Probability Field for Complex Occlusion Handling,” Proc. 10th Int'l Conf. Computer Vision, vol. 1, pp. 694-700, 2005.
[12] D. G-Perez, J.-M. Odobez, S. Ba, K. Smith, and G. Lathoud, “Tracking People in Meetings with Particles,” Proc. Int'l Workshop Image Analysis for Multimedia Interactive Services, 2005.
[13] D. Gavrila and V. Philomin, “Real-Time Object Detection for “Smart” Vehicles,” Proc. Seventh Int'l Conf. Computer Vision, vol. 1, pp. 87-93, 1999.
[14] P. Green, Trans-Dimensional Markov Chain Monte Carlo. Oxford Univ. Press, 2003.
[15] S. Haritaoglu, D. Harwood, and L. Davis, “W4: Real-Time Surveillance of People and Their Activities,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 809-830, Aug. 2000.
[16] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[17] W. Hasting, “Monte Carlo Sampling Methods Using Markov Chains and Their Applications,” Biometrika, vol. 57, no. 1, pp. 97-109, 1970.
[18] S. Hongeng and R. Nevatia, “Multi-Agent Event Recognition,” Proc. Eighth Int'l Conf. Computer Vision, vol. 2, pp. 84-91, 2001.
[19] M. Isard and J. MacCormick, “Bramble: A Bayesian Multiple-Blob Tracker,” Proc. Eighth Int'l Conf. Computer Vision, vol. 2, pp. 34-41, 2001.
[20] J. Kang, I. Cohen, and G. Medioni, “Continuous Tracking within and across Camera Streams,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 267-272, 2003.
[21] Z. Khan, T. Balch, and F. Dellaert, “MCMC-Based Particle Filtering for Tracking a Variable Number of Interacting Targets,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1805-1819, Nov. 2005.
[22] H.W. Kuhn, “The Hungarian Method for the Assignment Problem,” Naval Research Logistics Quarterly, vol. 2, pp. 83-87, 1955.
[23] M.-W. Lee and I. Cohen, “A Model-Based Approach for Estimating Human 3D Poses in Static Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp. 905-916, June 2006.
[24] A. Lipton, H. Fujiyoshi, and R. Patil, “Moving Target Classification and Tracking from Real-Time Video,” Proc. DARPA Image Understanding Workshop, pp. 129-136, 1998.
[25] J. Liu, “Metropolized Gibbs Sampler,” Monte Carlo Strategies in Scientific Computing, Springer, 2001.
[26] F. Lv, T. Zhao, and R. Nevatia, “Self-Calibration of a Camera from Video of a Walking Human,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1513-1518, Sept. 2006.
[27] J. MacCormick and A. Blake, “A Probabilistic Exclusion Principle for Tracking Multiple Objects,” Proc. Seventh Int'l Conf. Computer Vision, vol. 1, pp. 572-578, 1999.
[28] A. Mittal and L. Davis, “M2tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene Using Region-Based Stereo,” Proc. Seventh European Conf. Computer Vision, vol. 2, pp. 18-33, 2002.
[29] P. Nillius, J. Sullivan, and S. Carlsson, “Multi-Target Tracking-Linking Identities Using Bayesian Network Inference,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2187-2194, 2006.
[30] K. Okuma, A. Taleghani, N. de Freitas, J. Little, and D. Lowe, “A Boosted Particle Filter: Multitarget Detection and Tracking,” Proc. Eighth European Conf. Computer Vision, vol. 1, pp. 28-39, 2004.
[31] C. Papageorgiou, T. Evgeniou, and T. Poggio, “A Trainable Pedestrian Detection System,” Proc. IEEE Intelligent Vehicles Symp., pp. 241-246, 1998.
[32] A. Prati, I. Mikic, M. Trivedi, and R. Cucchiara, “Detecting Moving Shadows: Algorithms and Evaluation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 918-923, July 2003.
[33] P. Prez, C. Hue, J. Vermaak, and M. Gangnet, “Color-Based Probabilistic Tracking,” Proc. Seventh European Conf. Computer Vision, vol. 1, pp. 661-675, 2002.
[34] D. Ramanan, D. Forsyth, and A. Zisserman, “Strike a Pose: Tracking People by Finding Stylized Poses,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 271-278, 2005.
[35] C. Rasmussen and G.D. Hager, “Probabilistic Data Association Methods for Tracking Complex Visual Objects,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 560-576, June 2001.
[36] J. Rittscher, P. Tu, and N. Krahnstoever, “Simultaneous Estimation of Segmentation and Shape,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 487-493, 2005.
[37] R. Rosales and S. Sclaroff, “3D Trajectory Recovery for Tracking Multiple Objects and Trajectory Guided Recognition of Actions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp.2117-2123, 1999.
[38] H. Rue and MA. Hurn, “Bayesian Object Identification,” Biometrika, vol. 86, no. 3, pp. 649-660, 1999.
[39] C.R.H.S. Geman, “Diffusion for Global Optimization,” SIAM J. Control and Optimization, vol. 24, no. 5, pp. 1031-1043, 1986.
[40] N. Siebel and S. Maybank, “Fusion of Multiple Tracking Algorithms for Robust People Tracking,” Proc. Seventh European Conf. Computer Vision, vol. 4, pp. 373-387, 2002.
[41] K. Smith, D. Gatica-Perez, and J.-M. Odobez, “Using Particles to Track Varying Numbers of Interacting People,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 962-969, 2005.
[42] X. Song and R. Nevatia, “Combined Face-Body Tracking in Indoor Environment,” Proc. 17th Int'l Conf. Pattern Recognition, vol. 4, pp.159-162, 2004.
[43] X. Song and R. Nevatia, “A Model-Based Vehicle Segmentation Method for Tracking,” Proc. 10th Int'l Conf. Computer Vision, vol. 2, pp. 1124-1131, 2005.
[44] C. Stauffer and E. Grimson, “Learning Patterns of Activity Using Real-Time Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 747-757, Aug. 2000.
[45] C. Tao, H. Sawhney, and R. Kumar, “Object Tracking with Bayesian Estimation of Dynamic Layer Representations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp.75-89, Jan. 2002.
[46] H. Tao, H. Sawhney, and R. Kumar, “A Sampling Algorithm for Tracking Multiple Objects,” Proc. Workshop Vision Algorithms, 1999.
[47] L. Tierney, “Markov Chain Concepts Related to Sampling Algorithms,” Markov Chain Monte Carlo in Practice, pp. 59-74, 1996.
[48] Z.W. Tu and S.C. Zhu, “Image Segmentation by Data-Driven Markov Chain Monte Carlo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 651-673, May 2002.
[49] Y. Weiss, “Correctness of Local Probability Propagation in Graphical Models with Loops,” Neural Computation, vol. 12, no. 1, pp. 1-41, 2000.
[50] B. Wu and R. Nevatia, “Detection of Multiple, Partially Occluded Humans in a Single Image by Bayesian Combination of Edgelet Part Detectors,” Proc. 10th Int'l Conf. Computer Vision, vol. 1, pp.90-97, 2005.
[51] T. Yu and Y. Wu, “Collaborative Tracking of Multiple Targets,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp.834-841, 2004.
[52] T. Zhao, M. Aggarwal, R. Kumar, and H. Sawhney, “Real-Time Wide Area Multi-Camera Stereo Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 976-983, 2005.
[53] T. Zhao and R. Nevatia, “Bayesian Human Segmentation in Crowded Situations,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 459-466, 2003.
[54] T. Zhao and R. Nevatia, “Tracking Multiple Humans in Complex Situations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1208-1221, Sept. 2004.
[55] T. Zhao and R. Nevatia, “Tracking Multiple Humans in Crowded Environment,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 406-413, 2004.
[56] The CAVIAR Data Set, http://homepages.inf.ed.ac.uk/rbfCAVIAR/, 2008.
[57] CLEAR06 Evaluation Campaign and Workshop, http://isl.ira. uka.declear06/, 2008.
5 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool