This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Skin Color-Based Video Segmentation under Time-Varying Illumination
July 2004 (vol. 26 no. 7)
pp. 862-877

Abstract—A novel approach for real-time skin segmentation in video sequences is described. The approach enables reliable skin segmentation despite wide variation in illumination during tracking. An explicit second order Markov model is used to predict evolution of the skin-color (HSV) histogram over time. Histograms are dynamically updated based on feedback from the current segmentation and predictions of the Markov model. The evolution of the skin-color distribution at each frame is parameterized by translation, scaling, and rotation in color space. Consequent changes in geometric parameterization of the distribution are propagated by warping and resampling the histogram. The parameters of the discrete-time dynamic Markov model are estimated using Maximum Likelihood Estimation and also evolve over time. The accuracy of the new dynamic skin color segmentation algorithm is compared to that obtained via a static color model. Segmentation accuracy is evaluated using labeled ground-truth video sequences taken from staged experiments and popular movies. An overall increase in segmentation accuracy of up to 24 percent is observed in 17 out of 21 test sequences. In all but one case, the skin-color classification rates for our system were higher, with background classification rates comparable to those of the static segmentation.

[1] S.T. Birchfield, Elliptical Head Tracking Using Intensity Gradients and Color Histograms Proc. Conf. Computer Vision and Pattern Recognition, pp. 232-237, 1998.
[2] A. Blake and M. Isard, Active Contours. Cambridge Univ. Press 1998.
[3] A. Blake, M. Isard, and D. Reynard, Learning to Track the Visual-Motion of Contours Artificial Intelligence, vol. 78, nos. 1-2, pp. 179-212, 1995.
[4] T. Darrell, G. Gordon, M. Harville, and J. Woodfill, “Integrated Person Tracking Using Stereo, Color, and Pattern Detection,” Proc. Computer Vision and Pattern Recognition (CVPR '98), pp. 601-608, 1998.
[5] W. Hafner and O. Munkelt, Using Color for Detecting Persons in Image Sequences Pattern Recognition and Image Analysis, vol. 7, no. 1, pp. 47-52, 1997.
[6] M.J. Jones and J. M. Rehg, Statistical Color Models with Application to Skin Detection Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[7] N. Oliver, F. Bérard, and A. Pentland, “Lafter: Lips and Face Tracking,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR `97), June 1997.
[8] Y. Raja, S.J. McKenna, and S. Gong, Tracking and Segmenting People in Varying Lighting Conditions Using Colour Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 228-233, 1998.
[9] S. Singh and N. Papanikolopoulos, Vision-Based Detection of Driver Fatigue technical report, Dept. of Computer Science, Univ. of Minnesota, 1997.
[10] L. Sigal, S. Sclaroff, and V. Athitsos, Estimation and Prediction of Evolving Color Distributions for Skin Segmentation under Varying Illumination Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 152-159, 2000.
[11] C. Wren, A. Azarbayejani, T. Darrell, and A.P. Pentland, Pfinder: Real-Time Tracking of the Human Body IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 780-785, July 1997.
[12] J. Yang, L. Weier, and A. Waibel, Skin-Color Modeling and Adaptation Proc. Asian Conf. Computer Vision, vol. II, pp. 687-694, 1998.
[13] M.H. Yang and N. Ahuja, Detecting Human Faces in Color Images Proc. Int'l Conf. Image Processing, pp. 127-139, 1998.
[14] D. Saxe and R. Foulds, Toward Robust Skin Identification in Video Images Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 379-384, 1996.
[15] R. Kjeldsen and J. Kender, "Finding Skin in Color Images," Proc. Int'l Conf. Automatic Face and Gesture Recognition,Killington, Vt., pp. 312-317, Oct. 1996.
[16] M. Storring, H.J. Andersen, and E. Granum, Skin Colour Detection under Changing Lighting Conditions Proc. Seventh Symp. Intelligent Robotics Systems, pp. 187-195, 1999.
[17] S.-H. Kim, N.-K. Kim, S.C. Ahn, and H.-G. Kim, Object Oriented Face Detection Using Range and Color Information Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 76-81, 1998.
[18] J.C. Terrillon, M. David, and S. Akamatsu, “Automatic Detection of Human Faces in Natural Scene Images by Use of a Skin Color Model of Invariant Moments” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 112-117, 1998.
[19] K. Imagawa, S. Lu, and S. Igi, Color-Based Hands Tracking System for Sign Language Recognition Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 462-467, 1998.
[20] K. Schwerdt and J.L. Crowley, Robust Face Tracking Using Color Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 90-95, 2000.
[21] M. Storring, H.J. Andersen, and E. Granum, Estimation of the Illuminant Colour from Human Skin Colour Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 64-69, 2000.
[22] X. Zhu, J. Yang, and A. Waibel, Segmenting Hands of Arbitrary Color Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 446-453, 2000.
[23] J.C. Terrillon and S. Akamatsu, Comparative Performance of Different Chrominance Spaces for Color Segmentation and Detection of Human Faces in Complex Scene Images Proc. Vision Interface, pp. 180-187, 1999.
[24] M.H. Yang and N. Ahuja, Gaussian Mixture Model for Human Skin Color and Its Application in Image and Video Databases Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases, pp. 458-466, 1999.
[25] Y. Wu and T.S. Huang, Color Tracking by Transductive Learning Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp. 133-138, 1999.
[26] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning; Data Mining, Inference, and Prediction Springer Series in Statistics, Springer-Verlag, 2002.
[27] K. Fukunaga, Introduction to Statistical Pattern Recognition, second ed. Morgan Kaufmann, 1990.
[28] J.C. Terrillon, Y. Niwa, and K. Yamamoto, On the Selection of an Efficient Chrominance Space for Skin Color-Based Image Segmentation with an Application to Face Detection Proc. Int'l Conf. Quality Control by Artificial Vision, vol. 2, pp. 409-414, 2001.
[29] L. Bookstein, "Principal Warps: Thin-Plate Splines and the Decomposition of Deformations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 6, pp. 567-585, June 1989.
[30] R. Rosales, V. Athitsos, L. Sigal, and S. Sclaroff, 3D Hand Pose Reconstruction Using Specialized Mappings Proc. Int'l Conf. Computer Vision, vol. 1, pp. 378-385, 2001.
[31] M. Soriano, B. Martinkauppi, S. Huovinen, and M. Laaksonen, Skin Detection in Video under Changing Illumination Conditions Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 839-842, 2000.
[32] P. Smith, M. Shah, and N. Lobo, Monitoring Head/Eye Motion for Driver Alertness with One Camera Proc. Int'l Conf. Pattern Recognition, 2000.
[33] X. Liu, F. Xu, and K. Fujimura, Real-Time Eye Detection and Tracking for Driver Observation Under Various Light Conditions Proc. IEEE Intelligent Vehicle Symp., 2002.

Index Terms:
Color video segmentation, human skin detection, dynamic Markov model.
Citation:
Leonid Sigal, Stan Sclaroff, Vassilis Athitsos, "Skin Color-Based Video Segmentation under Time-Varying Illumination," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 7, pp. 862-877, July 2004, doi:10.1109/TPAMI.2004.35
Usage of this product signifies your acceptance of the Terms of Use.