This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Face Detection in Color Images
May 2002 (vol. 24 no. 5)
pp. 696-706

Human face detection plays an important role in applications such as video surveillance, human computer interface, face recognition, and face image database management. We propose a face detection algorithm for color images in the presence of varying lighting conditions as well as complex backgrounds. Based on a novel lighting compensation technique and a nonlinear color transformation, our method detects skin regions over the entire image and then generates face candidates based on the spatial arrangement of these skin patches. The algorithm constructs eye, mouth, and boundary maps for verifying each face candidate. Experimental results demonstrate successful face detection over a wide range of facial variations in color, position, scale, orientation, 3D pose, and expression in images from several photo collections (both indoors and outdoors).

[1] MPEG7 Content Set from Heinrich Hertz Institute, http://www.darmstadt.gmd.de/mobile/hm/projects/ MPEG7/DocumentsN2466.html. Oct. 1998.
[2] The Champion Database, http://www.libfind.unl.edu/alumni/events breakfast_for_champions.htm. Mar. 2001.
[3] Yahoo News Photos, http://dailynews.yahoo.com/h/gts/. Dec. 2001.
[4] M. Abdel-Mottaleb and A. Elgammal, “Face Detection in Complex Environments from Color Images,” IEEE Int'l Conf. Image Processing, pp. 622-626, 1999.
[5] L.M. Bergasa, M. Mazo, A. Gardel, M.A. Sotelo, and L. Boquete, “Unsupervised and Adaptive Gaussian Skin-Color Model,” Image and Vision Computing, vol. 18, pp. 987-1003, Sept. 2001.
[6] R. Brunelli and T. Poggio, "Face Recognition: Features vs. Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 10, pp. 1,042-1,053, Oct. 1993.
[7] R. Chellappa, C. Wilson, and S. Sirohey, "Human and Machine Recognition of Faces: A Survey," Proc. IEEE, vol. 83, no. 5, pp. 705-740, 1995.
[8] A.J. Colmenarez and T.S. Huang, Face Detection with Information-Based Maximum Discrimination Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 782-787, 1997.
[9] A. Colmenarez, B. Frey, and T. Huang, “Detection and Tracking of Faces and Facial Features,” IEEE Int'l Conf. Image Processing, pp. 657-661, Oct. 1999.
[10] D. DeCarlo and D. Metaxas, Optical Flow Constraints on Deformable Models with Applications to Face Tracking Int'l J. Computer Vision, vol. 38, no. 2, pp. 99-127, July 2000.
[11] R. Féraud, O.J. Bernier, J.-E. Viallet, and M. Collobert, “A Fast and Accurate Face Detection Based on Neural Network,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 1, pp. 42-53, Jan. 2001.
[12] C. Garcia and G. Tziritas, “Face Detection Using Quantized Skin Color Regions Merging and Wavelet Packet Analysis,” IEEE Trans. Multimedia, vol. 1, no. 3, pp. 264-277, Sept. 1999.
[13] M. Grudin, “On Internal Representation in Face Recognition Systems,” Pattern Recognition, vol. 33, pp. 1161-1177, 2000.
[14] E. Hjelm and B.K. Low, “Face Detection: A Survey,” Computer Vision and Image Understanding, vol. 83, no. 3, pp. 236-274, Sept. 2001.
[15] T. Horprasert, Y. Yacoob, and L.S. Davis, “Computing 3-D Head Orientation from a Monocular Image,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 242-247, Oct. 1996.
[16] R.-L. Hsu, M. Abdel-Mottaleb, and A.K. Jain, ”Face Detection in Color Images,” Techical Report MSU-CSE-01-7, Michigan State Univ., Mar. 2001.
[17] W. Huang, Q. Sun, C.-P. Lam, and J.-K. Wu, “A Robust Approach to Face and Eyes Detection from Images with Cluttered Background,” Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 110-114, Aug. 1998.
[18] P.T. Jackway and M. Deriche, “Scale-Space Properties of the Multiscale Morphological Dilation-Erosion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 1, pp. 38-51, Jan. 1996.
[19] M. Jones and J.M. Rehg, “Statistical Color Models with Application to Skin Detection,” Technical Report Series, Cambridge Research Laboratory, Dec. 1998.
[20] C. Kotropoulos, A. Tefas, and I. Pitas, "Frontal Face Authentication Using Morphological Elastic Graph Matching," IEEE Trans. Image Proc., vol. 9, no. 4, Apr. 2000, pp. 555-560.
[21] K.M. Lam and H. Yan, “An Analytic-to-Holistic Approach for Face Recognition Based on a Single Frontal View,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 7, pp. 673-686, July 1998.
[22] K.M. Lam and H. Yan, “Locating and Extracting the Eye in Human Face Images” Pattern Recognition, vol. 29, no. 5, pp. 771-779, 1996.
[23] M. Lew and N. Huijsmanns, "Information Theory and Face Detection," ICPR'96, pp. 601-605, 1996.
[24] D. Maio and D. Maltoni, “Real-Time Face Location on Gray-Scale Static Images,” Pattern Recognition, vol. 33, no. 9, pp. 1525-1539, Sept. 2000.
[25] B. Menser and M. Brunig, “Locating Human Faces in Color Images with Complex Background,” Intelligent Signal Processing and Comm. Systems, pp. 533-536, Dec. 1999.
[26] M. Pantic and L.J.M. Rothkrantz, “Automatic Analysis of Facial Expressions: The State of the Art,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424-1445, Dec. 1996.
[27] A. Pentland, B. Moghaddam, and Starner, "View-Based and Modular Eigenspaces for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994, pp. 84-91.
[28] P.J. Phillips, H. Moon, and S.A. Rozvi, The FERET Evaluation Methodolody for Face Recognition Algorithms IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090-1104, Oct. 2000.
[29] A. Nikolaidis and I. Pitas, “Facial Feature Extraction and Determination of Pose,” Pattern Recognition, vol. 33, pp. 1783-1791, 2000.
[30] E. Osuna, R. Freund, and F. Girosi, Training Support Vector Machines: An Application to Face Detection Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 17-19, 1997.
[31] C.A. Poynton, A Technical Introduction to Digital Video. John Wiley&Sons, 1996.
[32] H. Rowley, S. Baluja, and T. Kanade, "Neural Network-Based Face Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, Jan. 1998, pp. 23-38.
[33] H.A. Rowley, S. Baluja, and T. Kanade, “Rotation Invariant Neural Network-Based Face Detection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1998.
[34] E. Saber and A.M. Tekalp, “Frontal-View Face Detection and Facial Feature Extraction Using Color, Shape and Symmetry Based Cost Functions,” Pattern Recognition Letters, vol. 19, pp. 669-680, 1998.
[35] H. Schneiderman and T. Kanade, "A Statistical Method for 3D Object Detection Applied to Faces and Cars," Proc. IEEE Computer Vision and Pattern Recognition (CVPR 00), IEEE CS Press, 2000, pp. 746—751.
[36] A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, Content-Based Image Retrieval at the End of the Early Years IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, Dec. 2000.
[37] F. Smeraldi, O. Carmona, and J. Bigün, “Saccadic Search with Gabor Features Applied to Eye Detection and Real-time Head Tracking,” Image and Vision Computing, vol. 18, no. 4, pp. 323-329, 2000.
[38] K. Sobottka and I. Pitas, “A Novel Method for Automatic Face Segmentation, Facial Feature Extraction and Tracking,” Signal Processing: Image Comm., vol. 12, pp. 263-281, 1998.
[39] K.K. Sung and T. Poggio, "Example-Based Learning for View-Based Human Face Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 39-50, Jan. 1998.
[40] J.C. Terrillon, M.N. Shirazi, H. Fukamachi, and S. Akamatsu, “Comparative Performance of Different Skin Chrominance Models and Chrominance Spaces for the Automatic Detection of Human Faces in Color Images,” Proc. IEEE Int'l Conf. Face and Gesture Recognition, pp. 54-61, 2000.
[41] M. Turk and A. Pentland, "Face Recognition Using Eigenfaces," Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 1991, pp. 586-591.
[42] Face Recognition: From Theory to Applications, H. Wechsler, P. Phillips, V. Bruce, F. Soulie, and T. Huang, eds. Springer-Verlag, 1998.
[43] H. Wu, Q. Chen, and M. Yachida, “Face Detection from Color Images Using a Fuzzy Pattern Matching Method,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 6, pp. 557-563, June 1999.
[44] M.-H. Yang and N. Ahuja, ”Detecting Human Faces in Color Images,” Proc. IEEE Int'l Conf. Image Processing, pp. 127-139, Oct. 1998.
[45] M.-H. Yang, D.J. Kriegman, and N. Ahuja, "Detecting Faces in Images: A Survey," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, Jan. 2002, pp. 34-58.
[46] K.C. Yow and R. Cipolla, “Feature-Based Human Face Detection,” Image and Vision Computing, vol. 15, no. 9, pp. 713-735, Sept. 1997.
[47] W. Zhao, R. Chellappa, A. Rosenfeld, and P.J. Phillips, “Face Recognition: A Literature Survey,” CVL Technical Report, Center for Automation Research, Univ. of Maryland at College Park, ftp://ftp.cfar.umd.edu/TRs/CVL-Reports-2000 TR4167-zhao.ps.gz. Oct. 2000.

Index Terms:
Face detection, face recognition, lighting compensation, color transformation, skin tone, facial feature map, Hough transform
Citation:
R.L. Hsu, M. Abdel-Mottaleb, A.K. Jain, "Face Detection in Color Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 696-706, May 2002, doi:10.1109/34.1000242
Usage of this product signifies your acceptance of the Terms of Use.