The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - October (2011 vol.33)
pp: 1978-1990
Lior Wolf , Tel-Aviv University, Tel-Aviv
Tal Hassner , The Open University of Israel, Raanana
Yaniv Taigman , Tel-Aviv University, Tel-Aviv
ABSTRACT
Computer vision systems have demonstrated considerable improvement in recognizing and verifying faces in digital images. Still, recognizing faces appearing in unconstrained, natural conditions remains a challenging task. In this paper, we present a face-image, pair-matching approach primarily developed and tested on the “Labeled Faces in the Wild” (LFW) benchmark that reflects the challenges of face recognition from unconstrained images. The approach we propose makes the following contributions. 1) We present a family of novel face-image descriptors designed to capture statistics of local patch similarities. 2) We demonstrate how unlabeled background samples may be used to better evaluate image similarities. To this end, we describe a number of novel, effective similarity measures. 3) We show how labeled background samples, when available, may further improve classification performance, by employing a unique pair-matching pipeline. We present state-of-the-art results on the LFW pair-matching benchmarks. In addition, we show our system to be well suited for multilabel face classification (recognition) problem, on both the LFW images and on images from the laboratory controlled multi-PIE database.
INDEX TERMS
Face and gesture recognition, similarity measures, face recognition, image descriptors.
CITATION
Lior Wolf, Tal Hassner, Yaniv Taigman, "Effective Unconstrained Face Recognition by Combining Multiple Descriptors and Learned Background Statistics", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 10, pp. 1978-1990, October 2011, doi:10.1109/TPAMI.2010.230
REFERENCES
[1] L. Wolf, T. Hassner, and Y. Taigman, "Descriptor Based Methods in the Wild," Proc. Faces in Real-Life Images Workshop European Conf. Computer Vision, 2008.
[2] Y. Taigman, L. Wolf, and T. Hassner, "Multiple One-Shots for Utilizing Class Label Information," Proc. British Machine Vision Conf., Sept. 2009.
[3] L. Wolf, T. Hassner, and Y. Taigman, "Similarity Scores Based on Background Samples," Proc. Asian Conf. Computer Vision, Sept. 2009.
[4] L. Wolf, T. Hassner, and Y. Taigman, "The One-Shot Similarity Kernel," Proc. IEEE Int'l Conf. Computer Vision, Sept. 2009.
[5] G.B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, "Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments," Technical Report 07-49, Univ. of Massachusetts, Oct. 2007.
[6] N. Kumar, A. Berg, P. Belhumeur, and S. Nayar, "Attribute and Simile Classifiers for Face Verification," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[7] "LFW Results," www.cs.umass.edu/lfwresults.html, 2011.
[8] P. Phillips, H. Moon, P. Rauss, and S. Rizvi, "The FERET Evaluation Methodology for Face Recognition Algorithms," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, p. 1090, Oct. 2000.
[9] P. Phillips, P. Flynn, T. Scruggs, K. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, "Overview of the Face Recognition Grand Challenge," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 947-954, 2005.
[10] P. Phillips, P. Flynn, W. Scruggs, K. Bowyer, and W. Worek, "Preliminary Face Recognition Grand Challenge Results," Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 15-24, 2006.
[11] P. Phillips, W. Scruggs, A. OToole, P. Flynn, K. Bowyer, C. Schott, and M. Sharpe, "FRVT 2006 and ICE 2006 Large-Scale Results," Technical Report 7408, Nat'l Inst. Standard Technology Internal Reports (NISTIR), 2007.
[12] T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression Database," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[13] R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, "Multi-Pie," Image and Vision Computing, vol. 28, pp. 807-813, 2009.
[14] G. Huang, V. Jain, and E. Learned-Miller, "Unsupervised Joint Alignment of Complex Images," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[15] G.B. Huang, M.J. Jones, and E. Learned-Miller, "LFW Results Using a Combined Nowak Plus MERL Recognizer," Proc. Faces in Real-Life Images Workshop European Conf. Computer Vision, 2008.
[16] N. Pinto, J. DiCarlo, and D. Cox, "How Far Can You Get with a Modern Face Recognition Test Set Using Only Simple Features?," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[17] M. Guillaumin, J. Verbeek, C. Schmid, I. Lear, and L. Kuntzmann, "Is that you? Metric Learning Approaches for Face Identification," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[18] R. Brunelli and T. Poggio, "Face Recognition: Features versus Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 10, pp. 1042-1052, Oct. 1993.
[19] L. Wiskott, J.-M. Fellous, N. Krüger, and C. von der Malsburg, "Face Recognition by Elastic Bunch Graph Matching," IEEE Trans. Pattern Analysis and Machine Intelligence , vol. 19, no. 7, pp. 775-779, July 1997.
[20] E. Meyers and L. Wolf, "Using Biologically Inspired Features for Face Processing," Int'l J. Computer Vision, vol. 76, no. 1, pp. 93-104, 2008.
[21] T. Ojala, M. Pietikainen, and D. Harwood, "A Comparative-Study of Texture Measures with Classification Based on Feature Distributions," Pattern Recognition, vol. 29, no. 1, pp. 51-59, Jan. 1996.
[22] T. Ojala, M. Pietikäinen, and T. Mäenpää, "Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971-987, July 2002.
[23] T. Ojala, M. Pietikäinen, and T. Mäenpää, "A Generalized Local Binary Pattern Operator for Multiresolution Gray Scale and Rotation Invariant Texture Classification," Proc. Second Int'l Conf. Advances in Pattern Recognition, pp. 397-406, 2001.
[24] T. Ahonen, A. Hadid, and M. Pietikainen, "Face Description with Local Binary Patterns: Application to Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037-2041, Dec. 2006.
[25] M. Heikkilä, M. Pietikäinen, and C. Schmid, "Description of Interest Regions with Center-Symmetric Local Binary Patterns," Proc. Computer Vision, Graphics and Image Processing, Five Indian Conf., pp. 58-69, 2006.
[26] S. Belongie, J. Malik, and J. Puzicha, "Shape Context: A New Descriptor for Shape Matching and Object Recognition," Advances in Neural Information Processing Systems 13, T.K. Leen, T.G. Dietterich, and V. Tresp, eds., pp. 831-837, MIT Press, 2001.
[27] H. Zhang, A. Berg, M. Maire, and J. Malik, "SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[28] M. Bilenko, S. Basu, and R. Mooney, "Integrating Constraints and Metric Learning in Semi-Supervised Clustering," Proc. Int'l Conf. Machine Learning, 2004.
[29] N. Cristianini, J. Kandola, A. Elisseeff, and J. Shawe-Taylor, "On Kernel-Target Alignment," Proc. Neural Information Processing System, 2002.
[30] N. Shental, T. Hertz, D. Weinshall, and M. Pavel, "Adjustment Learning and Relevant Component Analysis," Proc. European Conf. Computer Vision, 2002.
[31] K. Weinberger, J. Blitzer, and L. Saul, "Distance Metric Learning for Large Margin Nearest Neighbor Classification," Proc. Neural Information Processing Systems, 2006.
[32] E. Xing, A.Y. Ng, M. Jordan, and S. Russell, "Distance Metric Learning, with Application to Clustering with Side-Information," Proc. Neural Information Processing Systems, 2003.
[33] L. Fei-Fei, R. Fergus, and P. Perona, "One-Shot Learning of Object Categories," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 594-611, Apr. 2006.
[34] M. Fink, "Object Classification from a Single Example Utilizing Class Relevance Pseudo-Metrics," Proc. Neural Information Processing Systems, 2004.
[35] O. Chapelle, B. Scholkopf, and A. Zien, Semi-Supervised Learning. MIT Press, 2006.
[36] T. Joachims, "Transductive Learning via Spectral Graph Partitioning," Proc. Int'l Conf. Machine Learning, pp. 290-297, 2003.
[37] A. Quattoni, M. Collins, and T. Darrell, "Transfer Learning for Image Classification with Sparse Prototype Representations," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.
[38] A. Bar-Hillel, T. Hertz, N. Shental, and D. Weinshall, "Learning Distance Functions Using Equivalence Relations," Proc. Int'l Conf. Machine Learning, 2003.
[39] W. Liu, S. Hoi, and J. Liu, "Output Regularized Metric Learning with Side Information," Proc. European Conf. Computer Vision, pp. 358-371, 2008.
[40] G. Chechik and N. Tishby, "Extracting Relevant Structures with Side Information," Proc. Neural Information Processing Systems, pp. 857-864, 2002.
[41] P. Jain, B. Kulis, and K. Grauman, "Fast Image Search for Learned Metrics," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[42] P. Jain, B. Kulis, and K. Grauman, "Fast Similarity Search for Learned Metrics," Technical Report #TR-07-48, Univ. of Texas, Sept. 2007.
[43] E. Nowak and F. Jurie, "Learning Visual Similarity Measures for Comparing Never Seen Objects," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2007.
[44] S. Ullman and E. Sali, "Object Classification Using a Fragment-Based Representation," Proc. First IEEE Int'l Workshop Biologically Motivated Computer Vision, pp. 73-87, 2000.
[45] L. Wolf, X. Huang, I. Martin, and D. Metaxas, "Patch-Based Texture Edges and Segmentation," Proc. European Conf. Computer Vision, pp. 481-493, 2006.
[46] E. Shechtman and M. Irani, "Matching Local Self-Similarities across Images and Videos," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2007.
[47] X. Tan and B. Triggs, "Enhanced Local Texture Feature Sets for Face Recognition under Difficult Lighting Conditions," Proc. Int'l Conf. Analysis and Modeling of Faces and Gestures, 2007.
[48] D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[49] S. Ebert, D. Larlus, and B. Schiele, "Extracting Structures in Image Collections for Object Recognition," Proc. European Conf. Computer Vision, 2010.
[50] M. Everingham, L. Van Gool, C. Williams, J. Winn, and A. Zisserman, "The PASCAL VOC Challenge 2008 Results," Proc. European Conf. Computer Vision, 2008.
[51] R. Fisher, "The Use of Multiple Measurements in Taxonomic Problems," Annals Eugenics, vol. 7, pp. 179-188, 1936.
[52] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. Springer, 2001.
[53] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. Wiley, 2001.
[54] A. Bordes and L. Bottou, "The Huller: A Simple and Efficient Online Svm," Proc. European Conf. Machine Learning, 2005.
[55] M. Brand, "Fast Low-Rank Modifications of the Thin Singular Value Decomposition," Linear Algebra and Its Applications, vol. 415, no. 1, pp. 20-30, 2006.
[56] S. Edelman, Representation and Recognition in Vision. MIT Press, 1999.
[57] E. Bart and S. Ullman, "Single-Example Learning of Novel Classes Using Representation by Similarity," Proc. British Machine Vision Conf., 2005.
[58] M. Riesenhuber and T. Poggio, "Hierarchical Models of Object Recognition in Cortex," Nature Neuroscience, vol. 2, no. 11, pp. 1019-1025, 1999.
[59] M. Roschani, "Evaluation of Local Descriptors on the Labeled Faces in the Wild Dataset," face.cs.kit.edu21_104.php, 2009.
[60] D.H. Wolpert, "Stacked Generalization," Neural Networks, vol. 5, no. 2, pp. 241-259, 1992.
[61] J. Davis, B. Kulis, P. Jain, S. Sra, and I. Dhillon, "Information-Theoretic Metric Learning," Proc. Int'l Conf. Machine Learning, pp. 209-216, June 2007.
25 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool