The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - November (2010 vol.32)
pp: 2022-2038
Liya Ding , The Ohio State University, Columbus
Aleix M. Martinez , The Ohio State University, Columbus
ABSTRACT
The appearance-based approach to face detection has seen great advances in the last several years. In this approach, we learn the image statistics describing the texture pattern (appearance) of the object class we want to detect, e.g., the face. However, this approach has had limited success in providing an accurate and detailed description of the internal facial features, i.e., eyes, brows, nose, and mouth. In general, this is due to the limited information carried by the learned statistical model. While the face template is relatively rich in texture, facial features (e.g., eyes, nose, and mouth) do not carry enough discriminative information to tell them apart from all possible background images. We resolve this problem by adding the context information of each facial feature in the design of the statistical model. In the proposed approach, the context information defines the image statistics most correlated with the surroundings of each facial component. This means that when we search for a face or facial feature, we look for those locations which most resemble the feature yet are most dissimilar to its context. This dissimilarity with the context features forces the detector to gravitate toward an accurate estimate of the position of the facial feature. Learning to discriminate between feature and context templates is difficult, however, because the context and the texture of the facial features vary widely under changing expression, pose, and illumination, and may even resemble one another. We address this problem with the use of subclass divisions. We derive two algorithms to automatically divide the training samples of each facial feature into a set of subclasses, each representing a distinct construction of the same facial component (e.g., closed versus open eyes) or its context (e.g., different hairstyles). The first algorithm is based on a discriminant analysis formulation. The second algorithm is an extension of the AdaBoost approach. We provide extensive experimental results using still images and video sequences for a total of 3,930 images. We show that the results are almost as good as those obtained with manual detection.
INDEX TERMS
Face detection, facial feature detection, shape extraction, subclass learning, discriminant analysis, adaptive boosting, face recognition, American sign language, nonmanuals.
CITATION
Liya Ding, Aleix M. Martinez, "Features versus Context: An Approach for Precise and Detailed Detection and Delineation of Faces and Facial Features", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 11, pp. 2022-2038, November 2010, doi:10.1109/TPAMI.2010.28
REFERENCES
[1] M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Fully Automatic Facial Action Recognition in Spontaneous Behavior," Proc. IEEE Conf. Face and Gesture, pp. 223-228, 2006.
[2] P.L. Bartlett and M. Traskin, "AdaBoost Is Consistent," J. Machine Learning Research, vol. 8, pp. 2347-2368, 2007.
[3] G. Carneiro, F. Amat, B. Georgescu, S. Good, and D. Comaniciu, "Semantic-Based Indexing of Fetal Anatomies from 3-D Ultrasound Data Using Global/Semi-Local Context and Sequential Sampling," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[4] T.F. Cootes, C.J. Taylor, D.H. Cooper, and J. Graham, "Active Shape Models—Their Training and Application," Computer Vision and Image Understanding, vol. 61, pp. 38-59, 1995.
[5] T.F. Cootes, G.J. Edwards, and C.J. Taylor, "Active Appearance Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-685, June 2001.
[6] D. Cristinacce and T. Cootes, "Automatic Feature Localisation with Constrained Local Models," Pattern Recognition, vol. 41, no. 10, pp. 3054-3067, 2008.
[7] F. De la Torre, J. Campoy, Z. Ambadar, and J.F. Cohn, "Temporal Segmentation of Facial Behavior," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[8] F. De la Torre and M.H. Nguyen, "Parameterized Kernel Principal Component Analysis: Theory and Applications to Supervised and Unsupervised Image Alignment," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[9] L. Ding and A.M. Martinez, "Precise Detailed Detection of Faces and Facial Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[10] P. Ekman and W.V. Friesen, The Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, 1978.
[11] S. Escalera, D.M.J. Tax, O. Pujol, P. Radeva, and R.P.W. Duin, "Subclass Problem Dependent Design for Error-Correcting Output Codes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 6, pp. 1041-1054, June 2008.
[12] T. Ezzat and T. Poggio, "Visual Speech Synthesis by Morphing Viseme," Int'l J. Computer Vision, vol. 38, no. 1, pp. 45-57, 2000.
[13] Y. Freund and R.E. Schapire, "A Decision Theoretic Generalization of Online Learning and an Application to Boosting," J. Computer and System Sciences, vol. 55, no. 1, pp. 119-139, 1995.
[14] L. Gu and T. Kanade, "A Generative Shape Regularization Model for Robust Face Alignment," Proc. European Conf. Computer Vision, Part I, pp. 413-426, 2008.
[15] B. Heisele, T. Serre, and T. Poggio, "A Component-Based Framework for Face Detection and Identification," Int'l J. Computer Vision, vol. 74, no. 2, pp. 167-181, 2007.
[16] R. Hsu, M. Abdel-Mottaleb, and A.K. Jain, "Face Detection in Color Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 696-705, May 2002.
[17] R.E. Kalman, "A New Approach to Linear Filtering and Prediction Problems," Trans. ASME J. Basic Eng., vol. 82D, no. 1, pp. 35-45, 1960.
[18] A. Lapedriza, D. Masip, and J. Vitria, "On the Use of Independent Tasks for Face Recognition," Proc. IEEE Conf. Pattern Recognition and Computer Vision, 2008.
[19] S.Z. Li and Z. Zhang, "FloatBoost Learning and Statistical Face Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1112-1123, Sept. 2004.
[20] P. Li and S.J.D. Prince, "Joint and Implicit Registration for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[21] L. Liang, F. Wen, X. Tang, and Y. Xu, "An Integrated Model for Accurate Shape Alignment," Proc. European Conf. Computer Vision, pp. 333-346, 2006.
[22] L. Liang, R. Xiao, F. Wen, and J. Sun, "Face Alignment via Component-Based Discriminative Searching," Proc. European Conf. Computer Vision, Part II, pp. 72-85, 2008.
[23] X. Liu, "Discriminative Face Alignment," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 11, pp. 1941-1954, Nov. 2009.
[24] E. Mäkinen and R. Raisamo, "Evaluation of Gender Classification Methods with Automatically Detected and Aligned Faces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp. 541-547, Mar. 2008.
[25] I.L. Dryden and K.V. Mardia, Statistical Shape Analysis. John Wiley, 1998.
[26] A.M. Martinez and R. Benavente, "The AR Face Database," Technical Report #24, Computer Vision Center (CVC), 1998.
[27] A.M. Martinez and A.C. Kak, "PCA versus LDA," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 228-233, Feb. 2001.
[28] A.M. Martinez, "Recognizing Imprecisely Localized, Partially Occluded and Expression Variant Faces from a Single Sample per Class," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp. 748-763, June 2002.
[29] A.M. Martinez, P. Mittrapiyanuruk, and A.C. Kak, "On Combining Graph-Partitioning with Non-Parametric Clustering for Image Segmentation," Computer Vision and Image Understanding, vol. 95, no. 1, pp. 72-85, 2004.
[30] A.M. Martinez and M. Zhu, "Where Are Linear Feature Extraction Methods Applicable?" IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1934-1944, Dec. 2005.
[31] M.S. Messing and R. Campbell, Gesture, Speech, and Sign. Oxford Univ. Press, 1999.
[32] K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre, "The Extended M2VTS Database," Proc. Int'l Conf. Audio- and Video-Based Biometric Person Authentication, pp. 72-77, 1999.
[33] S. Milborrow and F. Nicolls, "Locating Facial Features with an Extended Active Shape Model," Proc. IEEE Conf. Computer Vision and Pattern Recognition, Part IV, pp. 504-513, 2008.
[34] H. Moon, R. Chellappa, and A. Rosenfeld, "Optimal Edge-Based Shape Detection," IEEE Trans. Image Processing, vol. 11, no. 11, pp. 1209-1226, Nov. 2002.
[35] T. Moriyama, T. Kanade, J. Xiao, and J.F. Cohn, "Meticulously Detailed Eye Region Model and Its Application to Analysis of Facial Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 5, pp. 738-752, May 2006.
[36] S. Paisitkriangkrai, C. Shen, and J. Zhang, "Fast Pedestrian Detection Using a Cascade of Boosted Covariance Features," IEEE Trans. Circuits and Systems for Video Technology, vol. 18, no. 8, pp. 1140-1151, Aug. 2008.
[37] I. Patras and E.R. Hancock, "Regression Tracking with Data Relevance Determination," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[38] S. Romdhani and T. Vetter, "3D Probabilistic Feature Point Model for Object Detection and Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[39] D. Ross, J. Lim, R.-S. Lin, and M.-H. Yang, "Incremental Learning for Robust Visual Tracking," Int'l J. Computer Vision, vol. 77, nos. 1-3, pp. 125-141, 2008.
[40] K. Sung and T. Poggio, "Example-Based Learning for View-Based Human Face Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 39-51, Jan. 1998.
[41] X. Tang, Z. Ou, T. Su, H. Sun, and P. Zhao, "Robust Precise Eye Location by Adaboost and SVM Techniques," Proc. Int'l Symp. Neural Networks, pp. 93-98, 2005.
[42] A. Torralba, "Contextual Priming for Object Detection," Int'l J. Computer Vision, vol. 53, no. 2, pp. 169-191, 2003.
[43] O. Tuzel, F. Porikli, and P. Meer, "Pedestrian Detection via Classification on Riemannian Manifolds," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1713-1727, Oct. 2008.
[44] P. Viola and M. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. I. 511-518, 2001.
[45] D. Vukadinovic and M. Pantic, "Fully Automatic Facial Feature Point Detection Using Gabor Feature Based Boosted Classifiers," Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, pp. 1692-1698, 2005.
[46] P. Wang, M.B. Green, Q. Ji, and J. Wayman, "Automatic Eye Detection and Its Validation," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshop, 2005.
[47] L. Wolf and S. Bileschi, "A Critical View of Context," Int'l J. Computer Vision, vol. 69, no. 2, pp. 251-261, 2006.
[48] L. Wolf, T. Hassner, and Y. Taigman, "Descriptor Based Methods in the Wild," Proc. European Conf. Computer Vision, Workshop Faces in Real-Life Images: Detection, Alignment, and Recognition, 2008.
[49] J.X. Wu, S.C. Brubaker, M.D. Mullin, and J.M. Rehg, "Fast Asymmetric Learning for Cascade Face Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp. 369-382, Mar. 2008.
[50] M.-H. Yang, D.J. Kriegman, and N. Ahuja, "Detecting Faces in Images: A Survey," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 34-58, Jan. 2002.
[51] M.-H. Yang, "Face Localization," Encyclopedia of Biometrics, Springer, 2009.
[52] A.L. Yuille, D.S. Cohen, and P.W. Hallinan, "Feature Extraction from Faces Using Deformable Templates," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 104-109, 1989.
[53] Z. Zeng, M. Pantic, G.I. Roisman, and T.S. Huang, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009.
[54] H.T. Zhao and P.C. Yuen, "Incremental Linear Discriminant Analysis for Face Recognition," IEEE Trans. Systems, Man, and Cybernetics-Part B, vol. 38, no. 1, pp. 210-221, Feb. 2008.
[55] S. Zhou and R. Chellappa, "Multiple-Exemplar Discriminate Analysis for Face Recognition," Proc. Int'l Conf. Pattern Recognition, 2004.
[56] M. Zhu and A.M. Martinez, "Subclass Discriminant Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 8, pp. 1274-1286, Aug. 2006.
26 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool