This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Face Detection and Tracking in a Video by Propagating Detection Probabilities
October 2003 (vol. 25 no. 10)
pp. 1215-1228

Abstract—This paper presents a new probabilistic method for detecting and tracking multiple faces in a video sequence. The proposed method integrates the information of face probabilities provided by the detector and the temporal information provided by the tracker to produce a method superior to the available detection and tracking methods. The three novel contributions of the paper are: 1) Accumulation of probabilities of detection over a sequence. This leads to coherent detection over time and, thus, improves detection results. 2) Prediction of the detection parameters which are position, scale, and pose. This guarantees the accuracy of accumulation as well as a continuous detection. 3) The representation of pose is based on the combination of two detectors, one for frontal views and one for profiles. Face detection is fully automatic and is based on the method developed by Schneiderman and Kanade. It uses local histograms of wavelet coefficients represented with respect to a coordinate frame fixed to the object. A probability of detection is obtained for each image position and at several scales and poses. The probabilities of detection are propagated over time using a Condensation filter and factored sampling. Prediction is based on a zero order model for position, scale, and pose; update uses the probability maps produced by the detection routine. The proposed method can handle multiple faces, appearing/disappearing faces as well as changing scale and pose. Experiments carried out on a large number of sequences taken from commercial movies and the Web show a clear improvement over the results of frame-based detection (in which the detector is applied to each frame of the video sequence).

[1] S.T. Birchfield, Elliptical Head Tracking Using Intensity Gradients and Color Histograms Proc. Conf. Computer Vision and Pattern Recognition, pp. 232-237, 1998.
[2] M.J. Black and Y. Yacoob, "Tracking and Recognizing Rigid and Non-Rigid Facial Motions Using Local Parametric Model of Image Motion," Proc. Int'l Conf. Computer Vision, pp. 374-381,Cambridge, Mass., 1995.
[3] M.C. Burl, T.K. Leung, and P. Perona, Face Localisation via Shape Statistics Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 154-159, 1995.
[4] D. Comaniciu, V. Ramesh, and P. Meer, Real-Time Tracking of Non-Rigid Objects Using Mean Shift Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 142-149, 2000.
[5] D. Decarlo and D. Metaxas, Deformable Model Based Face Shape and Motion Estimation Proc. Int'l Conf. Face and Gesture Recognition, 1996.
[6] G.J. Edwards, C.J. Taylor, and T.F. Cootes, “Learning to Identify and Track Faces in Image Sequences,” Proc. Third Int'l Conf. Automatic Face and Gesture Recognition, pp. 260-265, 1998.
[7] R.S. Feris, T.E. de Campos, and R.M. Cesar Junior, Detection and Tracking of Facial Features in Video Sequences Lecture Notes in Artificial Intelligence, pp. 197-206, 2000.
[8] Applied Optimal Estimation, A. Gelb, ed. MIT Press, 1992.
[9] S. Gong, S. McKenna, and J. Collins, An Investigation into Face Pose Distribution Proc. IEEE Int'l Conf. Face and Gesture Recognition, 1996.
[10] D.B. Graham and N.M. Allinson, Characterizing Virtual Eigensignatures for General Purpose Face Recognition Face Recognition: From Theory to Applications, H. Wechsler, P.J. Phillips, V. Bruce, F. Fogelman-Soulie, and T.S. Huang, eds., NATO ASI Series F, Computer and Systems Sciences, vol. 163, pp. 446-456, 1998.
[11] G. Hager and K. Toyama, X Vision: A Portable Substrate for Real-Time Vision Applications Computer Vision and Image Understanding, vol. 69, no. 1, pp. 23-37, 1998.
[12] I. Haritaoglu, D. Harwood, and L.S. Davis, “W4 - a Real Time System for Detection and Tracking People and their Parts,” Proc. Third Face and Gesture Recognition Conf., pp. 222-227, 1998.
[13] E. Hjelmas and B.K. Low, Face Detection: A Survey Computer Vision and Image Understanding, pp. 236-274, 2001.
[14] E. Petajan, H.P. Graf, T. Chen, and E. Cosatto, Locating Faces and Facial Parts Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 41-46, 1995.
[15] M. Isard and A. Blake, Condensation-Conditional Density Propagation for Visual Tracking Int'l J. Computer Vision, vol. 29, pp. 5-28, 1998.
[16] M. Kirby and L. Sirovich,“Application of Karhunen-Loève procedure for the characterization of human faces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 103-108, Jan. 1990.
[17] A. Lanitis, C.J. Taylor, and T.F. Cootes, “Automatic Interpretation and Coding of Face Images using Flexible Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 743-756, July 1997.
[18] S. Li, Q. Fu, L. Gu, B. Schölkopf, Y. Cheng, and H. Zhang, Kernel Machine Based Learning for Multi-View Face Detection and Pose Detection Proc. Eighth Int'l Conf. Computer Vision, vol. II, pp. 674-679, 2001.
[19] Z. Liu and Y. Wang, Face Detection and Tracking in Video Using Dynamic Programming Proc. Int'l Conf. Image Processing, 2000.
[20] J. MacCormick and A. Blake, A Probabilistic Exclusion Principle for Tracking Multiple Objects Proc. Int'l Conf. Computer Vision, 1995.
[21] K. Mikolajczyk, R. Choudhury, and C. Schmid, Face Detection in a Video Sequence A Temporal Approach Proc. Conf. Computer Vision and Pattern Recognition, pp. 96-101, 2001.
[22] B. Moghaddam and A. Pentland, “Probabilistic Visual Learning for Object Representation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 696-710, July 1997.
[23] E. Osuna, R. Freund, and F. Girosi, Training Support Vector Machines: An Application to Face Detection Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 17-19, 1997.
[24] Y. Raja, S.J. McKenna, and S. Gong, Tracking and Segmenting People in Varying Lighting Conditions Using Colour Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 228-233, 1998.
[25] H. Rowley, S. Baluja, and T. Kanade, "Neural Network-Based Face Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, Jan. 1998, pp. 23-38.
[26] H. Schneiderman and T. Kanade, "A Statistical Method for 3D Object Detection Applied to Faces and Cars," Proc. IEEE Computer Vision and Pattern Recognition (CVPR 00), IEEE CS Press, 2000, pp. 746—751.
[27] K. Schwerdt and J.L. Crowley, Robust Face Tracking Using Color Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 90-95, 2000.
[28] A.W. Senior, Recognizing Faces in Broadcast Video Proc. IEEE Workshop Real-Time Analysis and Tracking of Face and Gesture in Real-Time Systems, pp. 105-110, 1999.
[29] A. Shashua, “Algebraic Functions for Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 8, pp. 779-789, 1995.
[30] T. Sim and T. Kanade, Combining Models and Exemplars for Face Fecognition Proc. Workshop Models versus Exemplars in Computer Vision, 2001.
[31] K.K. Sung and T. Poggio, "Example-Based Learning for View-Based Human Face Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 39-50, Jan. 1998.
[32] J.C. Terrillon, M.N. Shirazi, H. Fukamachi, and S. Akamatsu, “Comparative Performance of Different Skin Chrominance Models and Chrominance Spaces for the Automatic Detection of Human Faces in Color Images,” Proc. IEEE Int'l Conf. Face and Gesture Recognition, pp. 54-61, 2000.
[33] K. Toyama and A. Blake, Probabilistic Tracking in a Metric Space Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 50-57, 2001.
[34] M. Turk and A. Pentland, "Face Recognition Using Eigenfaces," Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 1991, pp. 586-591.
[35] P. Viola and M. Jones, Rapid Object Detection Using a Boosted Cascade of Simple Features Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001.
[36] C. Wren, A. Azarbayejani, T. Darrell, and A.P. Pentland, Pfinder: Real-Time Tracking of the Human Body IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 780-785, July 1997.
[37] J. Yang and A. Waibel, Tracking Human Faces in Real Time Technical Report CMU-CS-95-210, School of Computer Science, Carnegie Mellon Univ., Pittsburgh, Pa., 1995.
[38] M.-H. Yang, D.J. Kriegman, and N. Ahuja, "Detecting Faces in Images: A Survey," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, Jan. 2002, pp. 34-58.

Index Terms:
Face tracking, face detection, Condensation filter, video.
Citation:
Ragini Choudhury Verma, Cordelia Schmid, Krystian Mikolajczyk, "Face Detection and Tracking in a Video by Propagating Detection Probabilities," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1215-1228, Oct. 2003, doi:10.1109/TPAMI.2003.1233896
Usage of this product signifies your acceptance of the Terms of Use.