The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.09 - September (2011 vol.33)
pp: 1758-1775
S. Sclaroff , Dept. of Comput. Sci., Boston Univ., Boston, MA, USA
V. Ablavsky , Dept. of Comput. Sci., Boston Univ., Boston, MA, USA
ABSTRACT
We propose a representation for scenes containing relocatable objects that can cause partial occlusions of people in a camera's field of view. In many practical applications, relocatable objects tend to appear often; therefore, models for them can be learned offline and stored in a database. We formulate an occluder-centric representation, called a graphical model layer, where a person's motion in the ground plane is defined as a first-order Markov process on activity zones, while image evidence is aggregated in 2D observation regions that are depth-ordered with respect to the occlusion mask of the relocatable object. We represent real-world scenes as a composition of depth-ordered, interacting graphical model layers, and account for image evidence in a way that handles mutual overlap of the observation regions and their occlusions by the relocatable objects. These layers interact: Proximate ground-plane zones of different model instances are linked to allow a person to move between the layers, and image evidence is shared between the observation regions of these models. We demonstrate our formulation in tracking pedestrians in the vicinity of parked vehicles. Our results compare favorably with a sprite-learning algorithm, with a pedestrian tracker based on deformable contours, and with pedestrian detectors.
INDEX TERMS
object tracking, image representation, Markov processes, depth-ordered graphical model layer, layered graphical model, partially occluded object tracking, relocatable object, occluder-centric representation, first-order Markov process, image evidence, proximate ground-plane zone, pedestrian tracking, sprite-learning algorithm, Computational modeling, Cameras, Graphical models, Target tracking, Three dimensional displays, Pattern analysis, Image representation, graphical models., Computer vision, image representation, tracking
CITATION
S. Sclaroff, V. Ablavsky, "Layered Graphical Models for Tracking Partially Occluded Objects", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 9, pp. 1758-1775, September 2011, doi:10.1109/TPAMI.2011.43
REFERENCES
[1] V. Ablavsky, A. Thangali, and S. Sclaroff, "Layered Graphical Models for Tracking Partially-Occluded Objects," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2008.
[2] M. Andriluka, S. Roth, and B. Schiele, "People-Tracking-by-Detection and People-Detection-by-Tracking," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2008.
[3] M. Andriluka, S. Roth, and B. Schiele, "Pictorial Structures Revisited: People Detection and Articulated Pose Estimation," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2009.
[4] S. Atev and N. Papanikolopoulos, "Multi-View 3D Vehicle Tracking with a Constrained Filter," Proc. IEEE Int'l Conf. Robotics and Automation, 2008.
[5] S.T. Birchfield, "KLT: An Implementation of the Kanade-Lucas-Tomasi Feature Tracker," http://www.ces.clemson.edu/~stbklt/, 2009.
[6] G. Bradski and A. Kaebler, Learning OpenCV. O'Reilly Media, 2008.
[7] D. Comaniciu, V. Ramesh, and P. Meer, "Real-Time Tracking of Non-Rigid Objects Using Mean Shift," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2000.
[8] H. Dahlkamp, H.-H. Nagel, A. Ottlik, and P. Reuter, "A Framework for Model-Based Tracking Experiments in Image Sequences," Int'l J. Computer Vision, vol. 73, pp. 139-157, 2007.
[9] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification. Wiley-Interscience, 2001.
[10] A.M. Elgammal and L.S. Davis, "Probabilistic Framework for Segmenting People under Occlusion," Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001.
[11] P.F. Felzenszwalb, R.B. Girshick, D. McAllester, and D. Ramanan, "Object Detection with Discriminatively Trained Part-Based Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1627-1645, Sept. 2010.
[12] F. Fleuret, J. Berclaz, R. Lengagne, and P. Fua, "Multicamera People Tracking with a Probabilistic Occupancy Map," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 267-282, Feb. 2008.
[13] W. Ge and R.T. Collins, "Marked Point Processes for Crowd Counting," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2009.
[14] D. Gutchess, V. Ablavsky, A. Thangali, S. Sclaroff, and M. Snorrason, "Video Surveillance of Pedestrians and Vehicles," Proc. SPIE Conf. Tracking, Pointing, and Laser Systems Technologies XXI, 2007.
[15] I. Haritaoglu, D. Harwood, and L. Davis, "W4: Real-Time Surveillance of People and Their Activities," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 809-830, Aug. 2000.
[16] D. Hoiem, A.N. Stein, A.A. Efros, and M. Hebert, "Recovering Occlusion Boundaries from a Single Image," Proc. 11th IEEE Int'l Conf. Computer Vision, 2007.
[17] M. Irani and P. Anandan, "A Unified Approach to Moving Object Detection in 2D and 3D Scenes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 6, pp. 577-589, June 1998.
[18] M. Isard and J. MacCormick, "BraMBLe: A Bayesian Multiple-Blob Tracker," Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001.
[19] A.D. Jepson, D.J. Fleet, and M.J. Black, "A Layered Motion Representation with Occlusion and Compact Spatial Support," Proc. European Conf. Computer Vision, 2002.
[20] N. Jojic and B.J. Frey, "Learning Flexible Sprites in Video Layers," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2001.
[21] J. Kang, I. Cohen, and G. Medioni, "Continuous Tracking within and across Camera Streams," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2003.
[22] S.M. Khan and M. Shah, "A Multiview Approach to Tracking People in Crowded Scenes Using a Planar Homography Constraint," Proc. European Conf. Computer Vision, 2006.
[23] S.M. Khan and M. Shah, "Tracking Multiple Occluding People by Localizing on Multiple Scene Planes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 3, pp. 505-519, Mar. 2009.
[24] F.R. Kschischang, B.J. Frey, and H.-A. Loeliger, "Factor Graphs and the Sum-Product Algorithm," IEEE Trans. Information Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001.
[25] M.P. Kumar, P. Torr, and A. Zisserman, "Learning Layered Motion Segmentations of Video," Int'l J. Computer Vision, vol. 76, pp. 301-319, 2008.
[26] B. Leibe, K. Schindler, N. Cornelis, and L.V. Gool, "Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1683-1698, Oct. 2008.
[27] B. Leibe, A. Leonardis, and B. Schiele, "Robust Object Detection with Interleaved Categorization and Segmentation," Int'l J. Computer Vision, vol. 77, pp. 259-289, 2007.
[28] M.J. Leotta and J.L. Mundy, "Predicting High Resolution Image Edges with a Generic, Adaptive, 3-D Vehicle Model," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2009.
[29] A. Leykin and R. Hammoud, "Robust Multi-Pedestrian Tracking in Thermal-Visible Surveillance Videos," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition Workshop Object Tracking Beyond the Visible Spectrum, 2006.
[30] Y. Li, L. Gu, and T. Kanade, "A Robust Shape Model for Multi-View Car Alignment," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2009.
[31] F. Lv, T. Zhao, and R. Nevatia, "Camera Calibration from Video of a Walking Human," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1513-1518, Sept. 2006.
[32] X. Ma and W.E.L. Grimson, "Edge-Based Rich Representation for Vehicle Classification," Proc. 10th IEEE Int'l Conf. Computer Vision, 2005.
[33] A. Mittal and L.S. Davis, "M2tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene," Int'l J. Computer Vision, vol. 51, pp. 189-203, 2003.
[34] A. Ottlik and H.-H. Nagel, "Initialization of Model-Based Vehicle Tracking in Video Sequences of Inner-City Intersections," Int'l J. Computer Vision, vol. 80, pp. 211-225, 2008.
[35] A. Pece, "Contour Tracking Based on Marginalized Likelihood Ratios," Image and Vision Computing, vol. 24, pp. 301-317, 2006.
[36] T. Pollard and J. Mundy, "Change Detection in 3D World," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007.
[37] F. Porikli, O. Tuzel, and P. Meer, "Covariance Tracking Using Model Update Based on Lie Algebra," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.
[38] C. Rasmussen and G.D. Hager, "Probabilistic Data Association Methods for Tracking Complex Visual Objects," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 560-576, June 2001.
[39] J. Renno, J. Orwell, and G. Jones, "Learning Surveillance Tracking Models for the Self-Calibrated Ground Plane," Proc. British Machine Vision Conf., 2002.
[40] J. Renno, D. Greenhill, J. Orwell, and G. Jones, "Occlusion Analysis: Learning and Utilising Depth Maps in Object Tracking," Image and Vision Computing, vol. 26, pp. 430-441, 2008.
[41] M.S. Ryoo and J.K. Aggarwal, "Observe-and-Explain: A New Approach for Multiple Hypotheses Tracking of Humans and Objects," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2008.
[42] S.M. Seitz and C.R. Dyer, "Photorealistic Scene Reconstruction by Voxel Coloring," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1997.
[43] A. Senior, A. Hampapur, Y.-L. Tan, L. Brown, S. Pankanti, and R. Bolle, "Appearance Models for Occlusion Handling," Image and Vision Computing, vol. 24, pp. 1233-1243, 2006.
[44] N. Siebel and S.J. Maybank, "Real-Time Tracking of Pedestrians and Vehicles," Proc. Second IEEE Int'l Workshop Performance Evaluation of Tracking and Surveillance, 2001.
[45] L. Sigal, "Continuous-State Graphical Models for Object Localization, Pose Estimation and Tracking," PhD dissertation, Brown Univ., 2008.
[46] L. Sigal and M. Black, "Measure Locally, Reason Globally: Occlusion-Sensitive Articulated Pose Estimation," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.
[47] K. Smith, S.O. Ba, J.-M. Odobez, and D. Gatica-Perez, "Tracking the Visual Focus of Attention for a Varying Number of Wandering People," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 7, pp. 1212-1229, July 2008.
[48] K. Smith, D. Gatica-Perez, J.-M. Odobez, and S. Ba, "Evaluating Multi-Object Tracking," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition Workshop Empirical, Evaluation Methods in Computer Vision, 2005.
[49] V. Takala and M. Pietikainen, "Multi-Object Tracking Using Color, Texture and Motion," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007.
[50] H. Tao, H. Sawhney, and R. Kumar, "Object Tracking with Bayesian Estimation of Dynamic Layer Representations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 75-89, Jan. 2002.
[51] M. Titsias, "Unsupervised Learning of Multiple Objects in Images," PhD dissertation, School of Informatics, Univ. of Edinburgh, 2005.
[52] Performance Evaluation for Tracking and Surveillance (PETS) 2001 Dataset, The Univ. of Reading, UK Std., http://www.cvg.cs. rdg.ac.uk/PETS2001pets2001-dataset.html , 2001.
[53] L. Vacchetti, V. Lepetit, and P. Fua, "Stable Real-Time 3D Tracking Using Online and Offline Information," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1385-1391, Oct. 2004.
[54] S. Vedula, P. Rander, H. Saito, and T. Kanade, "Modeling, Combining, and Rendering Dynamic Real-World Events from Image Sequences," Proc. Fourth Int'l Conf. Virtual Systems and Multimedia, 1998.
[55] V. Venkataraman, X. Fan, and G. Fan, "Integrated Target Tracking and Recognition Using Joint Appearance," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition Workshop Object Tracking and Classification Beyond the Visible Spectrum, 2008.
[56] R. Vezzani and R. Cucchiara, "Ad-Hoc: Appearance Driven Human Tracking with Occlusion Handling," Proc. British Machine Vision Conf. Workshop Tracking Humans for the Evaluation of their Motion in Image Sequences, 2008.
[57] J. Wang and E. Adelson, "Representing Moving Images with Layers," IEEE Trans. Image Processing, vol. 3, no. 5, pp. 625-638, Sept. 1994.
[58] B. White and M. Shah, "Automatically Tuning Background Subtraction Parameters Using Particle Swarm Optimization," Proc. IEEE Int'l Conf. Multimedia and Expo, 2007.
[59] J. Winn and J. Shotton, "The Layout Consistent Random Field for Recognizing and Segmenting Partially Occluded Objects," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.
[60] B. Wu and R. Nevatia, "Detection and Segmentation of Multiple, Partially Occluded Objects by Grouping, Merging, Assigning Part Detection Responses," Int'l J. Computer Vision, vol. 82, pp. 185-204, 2009.
[61] B. Wu and R. Nevatia, "Simultaneous Object Detection and Segmentation by Boosting Local Shape Feature Based Classifier," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007.
[62] J. Xing, H. Ai, and S. Lao, "Multi-Object Tracking through Occlusions by Local Tracklets Filtering and Global Tracklets Association with Detection Responses," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2009.
[63] M. Xu and T. Ellis, "Partial Observation vs. Blind Tracking through Occlusion," Proc. British Machine Vision Conf., 2002.
[64] Z. Yin and R. Collins, "On-the-Fly Object Modeling While Tracking," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007.
[65] T. Yu, Y. Wu, N.O. Krahnstoever, and P.H. Tu, "Distributed Data Association and Filtering for Multiple Target Tracking," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2008.
[66] Y. Zhou and H. Tao, "A Background Layer Model for Object Tracking through Occlusion," Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
[67] L. Zhu, J. Zhou, and J. Song, "Tracking Multiple Objects Through Occlusion with Online Sampling and Position Estimation," Pattern Recognition, vol. 41, pp. 2447-2460, 2008.
26 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool