This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Detecting Carried Objects from Sequences of Walking Pedestrians
June 2012 (vol. 34 no. 6)
pp. 1056-1067
D. Damen, Dept. of Comput. Sci., Univ. of Bristol, Bristol, UK
D. Hogg, Sch. of Comput., Univ. of Leeds, Leeds, UK
This paper proposes a method for detecting objects carried by pedestrians, such as backpacks and suitcases, from video sequences. In common with earlier work [14], [16] on the same problem, the method produces a representation of motion and shape (known as a temporal template) that has some immunity to noise in foreground segmentations and phase of the walking cycle. Our key novelty is for carried objects to be revealed by comparing the temporal templates against view-specific exemplars generated offline for unencumbered pedestrians. A likelihood map of protrusions, obtained from this match, is combined in a Markov random field for spatial continuity, from which we obtain a segmentation of carried objects using the MAP solution. We also compare the previously used method of periodicity analysis to distinguish carried objects from other protrusions with using prior probabilities for carried-object locations relative to the silhouette. We have reimplemented the earlier state-of-the-art method [14] and demonstrate a substantial improvement in performance for the new method on the PETS2006 data set. The carried-object detector is also tested on another outdoor data set. Although developed for a specific problem, the method could be applied to the detection of irregularities in appearance for other categories of object that move in a periodic fashion.

[1] C. Benabdelkader and L. Davis, "Detection of People Carrying Objects: A Motion-Based Recognition Approach," Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 378-384, 2002.
[2] B. Bose and E. Grimson, "Ground Plane Rectification by Tracking Moving Objects," Proc. IEEE Int'l Workshop Performance Evaluation of Tracking and Surveillance, 2004.
[3] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[4] A. Branca, M. Leo, G. Attolico, and A. Distante, "Detection of Objects Carried by People," Proc. Int'l Conf. Image Processing, vol. 3, pp. 317-320, 2002.
[5] C.H. Chuang, J.W. Hsieh, L.W. Tsai, S.Y. Chen, and K.C. Fan, "Carried Object Detection Using Ratio Histogram and Its Application to Suspicious Event Analysis," IEEE Trans. Circuits and Systems for Video Technology, vol. 19, no. 6, pp. 911-916, June 2009.
[6] R. Cutler and L. Davis, "View-Based Detection and Analysis of Periodic Motion," Proc. 14th Int'l Conf. Pattern Recognition, vol. 1, pp. 495-500, 1998.
[7] R. Cutler and L. Davis, "Robust Real-Time Periodic Motion Detection, Analysis, and Applications," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 781-796, Aug. 2000.
[8] J. Davis and A. Bobick, "The Representation and Recognition of Action Using Temporal Templates," Proc. IEEE Int'l Conf. Computer Vision, 1997.
[9] M. Dimitrijevic, V. Lepetit, and P. Fua, "Human Body Pose Detection Using Bayesian Spatio-Temporal Templates," Computer Vision and Image Understanding, vol. 104, no. 2, pp. 127-139, 2006.
[10] M. Everingham and J. Winn, "The PASCAL Visual Object Classes Challenge (VOC2007) Development Kit," technical report, 2007.
[11] Proc. IEEE Int'l Workshop Performance Evaluation of Tracking and Surveillance, J. Ferryman, ed., 2006.
[12] A. Fossati, M. Dimitrijevic, V. Lepetit, and P. Fua, "Bridging the Gap between Detection and Tracking for 3D Monocular Video-Based Motion Capture," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[13] N.M. Ghanem and L.S. Davis, "Human Appearance Change Detection," Proc. 14th Int'l Conf. Image Analysis and Processing, pp. 536-541, 2007.
[14] I. Haritaoglu, R. Cutler, D. Harwood, and L.S. Davis, "Backpack: Detection of People Carrying Objects Using Silhouettes," Proc. Seventh IEEE Int'l Conf. Computer Vision, vol. 1, pp. 102-107, 1999.
[15] I. Haritaoglu, D. Harwood, and L. Davis, "Hydra: Multiple People Detection and Tracking Using Silhouettes," Proc. IEEE Workshop Visual Surveillance, 1999.
[16] I. Haritaoglu, D. Harwood, and L.S. Davis, "${\rm W}^4$ : Real-Time Surveillance of People and Their Activities," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 809-830, Aug. 2000.
[17] W. Hu, M. Hu, X. Zhou, T. Tan, J. Lou, and S. Maybank, "Principal Axis-Based Correspondence between Multiple Cameras for People Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 663-671, Apr. 2006.
[18] C.-S. Lee and A. Elgammal, "Carrying Object Detection Using Pose Preserving Dynamic Shape Models," Proc. Conf. Articulated Motion and Deformable Objects, pp. 315-325, 2006.
[19] F. Lv, T. Zhao, and R. Nevatia, "Self-Calibration of a Camera from Video of a Walking Human," Proc. 16th Int'l Conf. Pattern Recognition, vol. 1, pp. 562-567, 2002.
[20] D. Magee, "Tracking Multiple Vehicles Using Foreground, Background and Motion Models," Proc. Workshop Statistical Methods in Video Processing, pp. 7-12, 2002.
[21] H. Nanda, C. Benabdelkedar, and L. Davis, "Modelling Pedestrian Shapes for Outlier Detection: A Neural Net Based Approach," Proc. Intelligent Vehicles Symp., pp. 428-433, 2003.
[22] R. Pflugfelder and H. Bischof, "Online Auto-Calibration in Man-Made Worlds," Proc. Digital Image Computing, Techniques and Applications, 2005.
[23] P.J. Rousseeuw, "Least Median of Squares Regression," J. Am. Statistical Assoc., vol. 79, no. 388, pp. 871-880, 1984.
[24] M. Swain and D. Ballard, "Color Indexing," Int'l J. Computer Vision, vol. 7, no. 1, pp. 11-32, 1991.
[25] D. Tao, X. Li, S.J. Maybank, and W. Xindong, "Human Carrying Status in Visual Surveillance," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[26] Z. Zhang, "Iterative Point Matching for Registration of Free-Form Curves and Surfaces," Int'l J. Computer Vision, vol. 13, no. 2, pp. 119-152, 1994.
[27] Z. Zhang, M. Li, and T. Tan, "Robust Automated Ground Plane Rectification Based on Moving Vehicles for Traffic Scene Surveillance," Proc. Int'l Conf. Image Processing, pp. 1364-1367, 2008.

Index Terms:
video signal processing,image motion analysis,image representation,image segmentation,image sequences,Markov processes,maximum likelihood estimation,object detection,random processes,irregularity detection,carried object detection,walking pedestrian,video sequence,motion representation,shape representation,foreground segmentation,unencumbered pedestrian,likelihood map,Markov random field,carried object segmentation,MAP solution,Legged locomotion,Trajectory,Correlation,Cameras,Noise,Detectors,Lattices,periodicity analysis.,Baggage detection,carried-objects detection,silhouette analysis,temporal templates,template matching
Citation:
D. Damen, D. Hogg, "Detecting Carried Objects from Sequences of Walking Pedestrians," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 6, pp. 1056-1067, June 2012, doi:10.1109/TPAMI.2011.205
Usage of this product signifies your acceptance of the Terms of Use.