The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.09 - September (2009 vol.31)
pp: 1567-1581
Matthew Toews , Harvard Medical School, Boston
Tal Arbel , McGill University, Montreal
ABSTRACT
This paper presents a novel framework for detecting, localizing, and classifying faces in terms of visual traits, e.g., sex or age, from arbitrary viewpoints and in the presence of occlusion. All three tasks are embedded in a general viewpoint-invariant model of object class appearance derived from local scale-invariant features, where features are probabilistically quantified in terms of their occurrence, appearance, geometry, and association with visual traits of interest. An appearance model is first learned for the object class, after which a Bayesian classifier is trained to identify the model features indicative of visual traits. The framework can be applied in realistic scenarios in the presence of viewpoint changes and partial occlusion, unlike other techniques assuming data that are single viewpoint, upright, prealigned, and cropped from background distraction. Experimentation establishes the first result for sex classification from arbitrary viewpoints, an equal error rate of 16.3 percent, based on the color FERET database. The method is also shown to work robustly on faces in cluttered imagery from the CMU profile database. A comparison with the geometry-free bag-of-words model shows that geometrical information provided by our framework improves classification. A comparison with support vector machines demonstrates that Bayesian classification results in superior performance.
INDEX TERMS
Scale-invariant feature, viewpoint invariance, probabilistic modeling, visual trait, sex classification, faces, occlusion.
CITATION
Matthew Toews, Tal Arbel, "Detection, Localization, and Sex Classification of Faces from Arbitrary Viewpoints and under Occlusion", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 9, pp. 1567-1581, September 2009, doi:10.1109/TPAMI.2008.233
REFERENCES
[1] E. Makinen and R. Raisamo, “Evaluation of Gender Classification Methods with Automatically Detected and Aligned Faces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp.541-547, Mar. 2008.
[2] S. Baluja and H.A. Rowley, “Boosting Sex Identification Performance,” Int'l J. Computer Vision, vol. 71, no. 1, pp. 111-119, 2007.
[3] Z. Yang, M. Li, and H. Ai, “An Experimental Study on Automatic Face Gender Classification,” Proc. Int'l Conf. Pattern Recognition, pp. 1099-1102, 2006.
[4] A. Jain, J. Huang, and S. Fang, “Gender Identification Using Frontal Facial Images,” Proc. IEEE Int'l Conf. Multimedia and Expo, pp. 1082-1085, 2005.
[5] B. Moghaddam and M. Yang, “Learning Gender with Support Faces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 707-711, May 2002.
[6] S. Gutta, H. Wechsler, and P. Phillips, “Gender and Ethnic Classification of Human Faces Using Hybrid Classifiers,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 194-199, 1998.
[7] M. Toews and T. Arbel, “Detection over Viewpoint via the Object Class Invariant,” Proc. Int'l Conf. Pattern Recognition, vol. 1, pp.765-768, 2006.
[8] A. Thomas, V. Ferrari, B. Leibe, T. Tuytelaars, B. Schiele, and L. Van Gool, “Towards Multi-View Object Class Detection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp.1589-1596, 2006.
[9] A. Kushal, C. Schmid, and J. Ponce, “Flexible Object Models for Category-Level 3D Object Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[10] S. Savarese and L. Fei-Fei, “3D Generic Object Categorization, Localization and Pose Estimation,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[11] P. Yan, S.M. Khan, and M. Shah, “3D Model Based Object Class Detection in an Arbitrary View,” Proc. IEEE Int'l Conf. Computer Vision, 2007.
[12] CMU Face Group “Frontal and Profile Face Databases,” http://vasc.ri.cmu.edu/idb/htmlface/, 2009.
[13] “Color FERET Face Database,” www.itl.nist.gov/iad/humanidcolorferet, 2009.
[14] M. Toews and T. Arbel, “Detecting, Localizing and Classifying Visual Traits from Arbitrary Viewpoints Using Probabilistic Local Feature Modeling,” Proc. IEEE Workshop Analysis and Modeling of Faces and Gestures, pp. 154-167, 2007.
[15] M. Turk and A.P. Pentland, “Eigenfaces for Recognition,” J.Cognitive Neuroscience, vol. 3, no. 1, pp. 71-96, 1991.
[16] S. Ullman, M. Vidal-Naquet, and E. Sali, “Visual Features of Intermediate Complexity and Their Use in Classification,” Nature Neuroscience, vol. 5, pp. 682-687, 2002.
[17] B. Heisele, T. Serre, and T. Poggio, “A Component-Based Framework for Face Detection and Identification,” Int'l J. Computer Vision, vol. 74, no. 2, pp. 167-181, 2007.
[18] P. Viola and M. Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 511-518, 2001.
[19] M. Jones and P. Viola, “Fast Multi-View Face Detection,” Technical Report tr2003-96, pp. 1-10, 2003.
[20] D.G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[21] G. Carneiro and A. Jepson, “Multi-Scale Phase-Based Local Features,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 736-743, 2003.
[22] K. Mikolajczyk and C. Schmid, “Indexing Based on Scale Invariant Interest Points,” Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp.525-531, 2001.
[23] T. Kadir and M. Brady, “Saliency, Scale and Image Description,” Int'l J. Computer Vision, vol. 45, no. 2, pp. 83-105, 2001.
[24] F. Mindru, T. Tuytelaars, L. Van Gool, and T. Moons, “Moment Invariants for Recognition under Changing Viewpoint and Illumination,” Computer Vision and Image Understanding, vol. 94, nos. 1-3, pp. 3-27, 2004.
[25] H. Schneiderman and T. Kanade, “Object Detection Using the Statistics of Parts,” Int'l J. Computer Vision, vol. 56, no. 3, pp. 155-177, 2004.
[26] R. Fergus, P. Perona, and A. Zisserman, “Weakly Supervised Scale-Invariant Learning of Models for Visual Recognition,” Int'l J. Computer Vision, vol. 71, no. 3, pp. 273-303, 2006.
[27] G. Dorko and C. Schmid, “Selection of Scale-Invariant Parts for Object Class Recognition,” Proc. IEEE Int'l Conf. Computer Vision, pp. 634-640, 2003.
[28] J. Sivic, B.C. Russell, A.A. Efros, A. Zisserman, and W.T. Freeman, “Discovering Objects and Their Localization in Images,” Proc. IEEE Int'l Conf. Computer Vision, pp. 370-377, 2005.
[29] D. Crandal, P. Felzenswalb, and D. Huttenlocher, “Spatial Priors for Part-Based Recognition Using Statistical Models,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp.10-17, 2005.
[30] E. Bart, E. Byvatov, and S. Ullman, “View-Invariant Recognition Using Corresponding Object Fragments,” Proc. European Conf. Computer Vision, pp. 152-165, 2004.
[31] P.F. Felzenszwalb and D. Huttenlocher, “Pictorial Structures for Object Recognition,” Int'l J. Computer Vision, vol. 61, no. 1, pp. 55-79, 2005.
[32] G. Carneiro and D.G. Lowe, “Sparse Flexible Models of Local Features,” Proc. European Conf. Computer Vision, vol. 3, pp.29-43, 2006.
[33] E. Sudderth, A. Torralba, W. Freeman, and A. Willsky, “Describing Visual Scenes Using Transformed Objects and Parts,” Int'l J. Computer Vision, vol. 77, nos. 1-3, pp. 291-330, 2008.
[34] B. Leibe, A. Leonardis, and B. Schiele, “Combined Object Categorization and Segmentation with an Implicit Shape Model,” Proc. ECCV Workshop Statistical Learning in Computer Vision, 2004.
[35] M. Weber, M. Welling, and P. Perona, “Unsupervised Learning of Models for Recognition,” Proc. European Conf. Computer Vision, vol. 1, pp. 18-32, 2000.
[36] I. Beiderman and P.C. Gerhardstein, “Recognizing Depth-Rotated Objects: Evidence and Conditions for 3D Viewpoint Invariance,” J.Experimental Psychology: Human Perception and Performance, vol. 19, pp. 1162-1182, 1993.
[37] J. Koenderink, “The Internal Representation of Solid Shape with Respect to Vision,” Biological Cybernetics, vol. 32, no. 4, pp. 211-216, 1979.
[38] J. Burns, R. Weiss, and E. Riseman, “The Non-Existence of General-Case View-Invariants,” Geometric Invariance in Computer Vision, MIT Press, pp. 120-131, 1992.
[39] I. Beiderman, “Recognition-by-Components: A Theory of Human Image Understanding,” Psychological Rev., vol. 94, no. 2, pp.115-147, 1987.
[40] J. Ponce, D. Chelberg, and W. Mann, “Invariant Properties of Straight Homogeneous Generalized Cylinders and Their Contours,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 9, pp. 951-966, Sept. 1989.
[41] R. Fergus, P. Perona, and A. Zisserman, “Object Class Recognition by Unsupervised Scale-Invariant Learning,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 264-271, 2003.
[42] N. Ahuja and S. Todorovic, “Learning the Taxonomy and Models of Categories Present in Arbitrary Images,” Proc. IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007.
[43] B.C. Russell, A.A. Efros, J. Sivic, W.T. Freeman, and A. Zisserman, “Using Multiple Segmentations to Discover Objects and Their Extent in Image Collections,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1605-1614, 2006.
[44] L. Fei-Fei, A. Iyer, C. Koch, and P. Perona, “What Do We Perceive in a Glance of a Real-World Scene?” J. Vision, vol. 7, no. 1, pp. 1-29, 2007.
[45] M. Everingham, A. Zisserman, C.K.I. Williams, and L. Van Gool, “The PASCAL Visual Object Classes Challenge 2006 (VOC2006) Results,” www.pascal-network.org/challenges/VOC/voc2006 results.pdf, 2009.
[46] B.C. Russell, A. Torralba, K.P. Murphy, and W.T. Freeman, “LabelMe: A Database and Web-Based Tool for Image Annotation,” Int'l J. Computer Vision, vol. 77, nos. 1-3, pp. 157-173, 2008.
[47] Motorcycle Safety Foundation Inc., Basic Rider Course Handbook, seventh ed., www.msf-usa.org/CurriculumMaterialsBRC_ Handbook_Vs71_noprint.pdf , 2007.
[48] H.-C. Kim, D. Kim, Z. Ghahramani, and S.Y. Bang, “Appearance-Based Gender Classification with Gaussian Processes,” Pattern Recognition Letters, vol. 27, pp. 618-626, 2006.
[49] A.J. O'Toole, T. Vetter, N.F. Troje, and H.H. Bulthoff, “Sex Classification Is Better with Three-Dimensional Structure than with Image Intensity Information,” Perception, vol. 26, pp. 75-84, 1997.
[50] C. BenAbdelkader and P. Griffin, “A Local Region-Based Approach to Gender Classification from Face Images,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[51] G. Shakhnarovich, P.A. Viola, and B. Moghaddam, “A Unified Learning Framework for Real Time Face Detection and Classification,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, 2002.
[52] Y. Lamdan, J.T. Schwartz, and H.J. Wolfson, “On Recognition of 3D Objects from 2D Images,” Proc. IEEE Int'l Conf. Robotics and Automation, pp. 1407-1413, 1988.
[53] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. Wiley, 2001.
[54] D. Comaniciu and P. Meer, “Mean Shift: A Robust Approach toward Feature Space Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[55] M. Toews and T. Arbel, “Detecting and Localizing 3D Object Classes Using Viewpoint Invariant Reference Frames,” Proc. Int'l Conf. Computer Vision (ICCV Workshop) 3D Representation for Recognition, 2007.
[56] M.I. Jordan, An Introduction to Probabilistic Graphical Models, in preparation.
[57] E. Jaynes, “Prior Probabilities,” IEEE Trans. Systems, Science, and Cybernetics, vol. 4, no. 3, pp. 227-241, Sept. 1968.
[58] C. Schmid, R. Mohr, and C. Bauckhage, “Evaluation of Interest Point Detectors,” Int'l J. Computer Vision, vol. 37, no. 2, pp. 151-172, June 2000.
[59] K. Mikolajczyk and C. Schmid, “A Performance Evaluation of Local Descriptors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, Oct. 2005.
[60] C. Dance, J. Willamowski, L. Fan, C. Bray, and G. Csurka, “Visual Categorization with Bags of Keypoints,” Proc. European Conf. Computer Vision (ECCV Workshop) Statistical Learning in Computer Vision, 2004.
[61] C. Cortes and V. Vapnik, “Support-Vector Networks,” Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.
[62] C.-C. Chang and C.-J. Lin, “LIBSVM—A Library for Support Vector Machines,” www.csie.ntu.edu.tw/cjlinlibsvm, 2007.
[63] M. Toews and T. Arbel, “A Statistical Parts-Based Appearance Model of Anatomical Variability,” IEEE Trans. Medical Imaging, special issue on computational neuroanatomy, vol. 26, no. 4, pp.497-508, Apr. 2007.
[64] A. Opelt, A. Pinz, M. Fussenegger, and P. Auer, “Generic Object Recognition with Boosting,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 416-431, Mar. 2006.
25 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool