| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
W4: Real-Time Surveillance of People and Their Activities
August 2000 (vol. 22 no. 8)
pp. 809-830
Abstract—$W^4$ is a real time visual surveillance system for detecting and tracking multiple people and monitoring their activities in an outdoor environment. It operates on monocular gray-scale video imagery, or on video imagery from an infrared camera. $W^4$ employs a combination of shape analysis and tracking to locate people and their parts (head, hands, feet, torso) and to create models of people's appearance so that they can be tracked through interactions such as occlusions. It can determine whether a foreground region contains multiple people and can segment the region into its constituent people and track them. $W^4$ can also determine whether people are carrying objects, and can segment objects from their silhouettes, and construct appearance models for them so they can be identified in subsequent frames. $W^4$ can recognize events between people and objects, such as depositing an object, exchanging bags, or removing an object. It runs at 25 Hz for 320$\times$240 resolution images on a 400 Mhz dual-Pentium II PC.
[1] 809 K. Akita, “Image Sequence Analysis of Real World Human Motion, Pattern Recognition, vol. 17, no. 4, pp. 73-83, 1984.[2] A. Azarbayjani, C. Wren, and A. Pentland, “Real-Time 3D Tracking of the Human Body,” Proc. IMAGE'COM, 1996.[3] D. Beymer and K. Konolige, “Real-Time Tracking of Multiple People Using Stereo,” Proc. IEEE Frame Rate Workshop, 1999.[4] A. Bobick and J. Davis, “Real-Time Recognition of Activity Using Temporal Templates,” Proc. IEEE Workshop Application of Computer Vision, pp. 1,233-1,251, 1996.[5] A. Bobick, J. Davis, S. Intille, F. Baird, L. Cambell, Y. Irinov, C. Pinhanez, and A. Wilson., “Kidsroom: Action Recognition in an Interactive Story Environment,” Technical Report 398, M.I.T. Perceptual Computing, 1996.[6] T. Boult, “Frame-Rate Multibody Tracking for Surveillance,” Proc. DARPA Image Understanding Workshop, 1998.[7] C. Bregler and J. Malik, “Tracking People with Twists and Exponential Maps,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 8–15, June 1998.[8] R. Cutler and L. Davis, “View-Based Detection and Analysis of Periodic Motion,” Proc. Int'l Conf. Pattern Recognition, 1998.[9] T. Darell, G. Gordon, M. Harville, J. Woodfill, “Integrated Person Tracking Using Stereo, Color, and Pattern Detection,” Computer Vision and Pattern Recognition, 1998.[10] A. Elgammal, D. Harwood, and L. Davis, “Non-Parametric Model for Background Subtraction,” Proc. IEEE Frame Rate Workshop, 1999.[11] N. Friedman and S. Russell, “Image Segmentation in Video Sequences: A Probabilistic Approach,” Uncertainty in Artificial Intelligence, 1997.[12] D.M. Gavrila, “The Visual Analysis of Human Movement: A Survey,” Computer Vision and Image Understanding, vol. 73, no. 1, Jan. 1999.[13] W.E.L. Grimson, L. Lee, R. Romano, and C. Stauffer, “Using Adaptive Tracking to Classify and Monitor Activities in a Site,“ IEEE Proc. Computer Vision and Pattern Recognition, pp. 22-31, 1998.[14] E. Grimson and C. Stauffer, “Adaptive Background Mixture Models for Real Time Tracking,“ Proc. Computer Vision and Pattern Recognition Conf., 1999.[15] I. Haritaoglu, D. Harwood, and L.S. Davis, “W4 - a Real Time System for Detection and Tracking People and their Parts,” Proc. Third Face and Gesture Recognition Conf., pp. 222-227, 1998.[16] I. Haritaoglu, D. Harwood, and L.S. Davis, “W4S: A Real-Time System for Detecting and Tracking People in 2 1/2-D,” Proc. European Conf. Computer Vision, 1998.[17] I. Haritaoglu, D. Harwood, and L. Davis, “Ghost: A Human Body Part Labeling System Using Silhouettes,” Proc. Int'l Conf. Pattern Recognition, 1998.[18] I. Haritaoglu, D. Harwood, and L. Davis, “Backpack: Detecting People Carrying Object Using Silhouettes,” Proc. Int'l Conf. Computer Vision, 1999.[19] I. Haritaoglu, “W4: A Real Time System for Detection and Tracking of People and Monitoring Their Activities,” PhD thesis, University of Maryland, Computer Science Dept., 1999.[20] T. Horprasert, I. Haritaoglu, D. Harwood, L. Davis, C. Wren, and A. Pentland, “Real-Time 3D Motion Capture,” Proc. Second Workshop Perceptual Interfaces, Nov. 1998.[21] T. Horprasert, D. Harwood, and L.S. Davis, “A Robust Background Subtraction and Shadow Detection,” Proc. Asian Conf. Computer Vision, Jan. 2000.[22] S.S. Intille, J.W. Davis, and A.F. Bobick, “Real Time Closed World Tracking,” IEEE Proc. Computer Vision and Pattern Recognition, pp. 697-703, 1997.[23] S. Iwasawa, K. Ebihara, J. Ohya, and S. Morishima, “Real-Time Estimation of Human Body Posture from Monocular Thermal Images,“ Proc. Computer Vision and Pattern Recognition, 1997.[24] K. Konolige, “Small Vision Systems: Hardware and Implementation,“ Proc. Int'l Symp. Robotics Research, 1997.[25] A.J. Lipton, H. Fujiyoshi, and R.S. Patil, “Moving Target Classification and Tracking from Real Time Video,” Proc. Fourth IEEE Workshop Applications of Computer Vision '98, pp. 8-14, 1998.[26] M. Leung and Y.H. Yang, “A Model Based Approach to Labeling Human Body Outlines,” 1994.[27] R Morris and D. Hogg, “Statistical Models of Object Interactions,” Proc. IEEE Workshop Visual Surveillance, 1998.[28] N. Oliver, B. Rosario, and A. Pentland, “Statistical Modeling of Human Interactions,” Proc. Workshop Interpretation of Visual Motion, pp. 8-15, 1998.[29] T. Olson and F. Brill, “Moving Object Detection and Event Recognition Algorithms for Smart Cameras,” Proc. DARPA Image Understanding Workshop, pp. 159-175, 1997.[30] J. O'Rourke and N. Badler, “Model-Based Image Analysis of Human Motion Using Constraint Propagation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 2, pp. 522-536, 1980.[31] J. Rehg, M. Loughlin, and K. Waters, “Vision for a Smart Kiosk,” Computer Vision and Pattern Recognition, 1997.[32] J. Segen and S. Pingali, “A Camera-Based System for Tracking People in Real-Time,” Proc. Int'l Conf. Computer Vision,” 1996.[33] Y.Y. Shanon, X. Ju, and M.J. Black, “Cardboard People: A Parameterized Model of Articulated Image Motion,” Proc. Second Int'l Conf. Automatic Face and Gesture Recognition, 1996.[34] C. Wren, A. Azarbayejani, T. Darrell, and A.P. Pentland, Pfinder: Real-Time Tracking of the Human Body IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 780-785, July 1997.[35] C. Wren and A. Pentland, “Dynamic Modeling of Human Motion,” Proc. IEEE Conf. Automatic Face and Gesture Recognition, pp. 22-27, Nara, Japan, Apr. 1998.[36] A. Selinger and L. Wixson, “Classfying Moving Object as Rigid or Non-Rigid without Correspondences,” Proc. DARPA Image Understanding Workshop,” 1998.[37] A. Shafer, J. Krumm, B Brumitt, B. Meyers, M. Czerwinski, and D. Robbins, “The New EasyLiving Project at Microsoft,” Proc. DARPA/NIST Smart Spaces Workshop, 1998.
Index Terms:
Surveillance, people tracking, activity detection, real-time vision, body part analysis.
Citation:
Ismail Haritaoglu, David Harwood, Larry S. Davis, "W4: Real-Time Surveillance of People and Their Activities," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 809-830, Aug. 2000, doi:10.1109/34.868683
