The Community for Technology Leaders
Subscribe
Issue No.05 - May (2013 vol.35)
pp: 1193-1205
Shengcai Liao , Center for Biometrics & Security Res., Inst. of Autom., Beijing, China
A. K. Jain , Dept. of Comput. Sci. & Eng., Michigan State Univ., East Lansing, MI, USA
S. Z. Li , Center for Biometrics & Security Res., Inst. of Autom., Beijing, China
ABSTRACT
Numerous methods have been developed for holistic face recognition with impressive performance. However, few studies have tackled how to recognize an arbitrary patch of a face image. Partial faces frequently appear in unconstrained scenarios, with images captured by surveillance cameras or handheld devices (e.g., mobile phones) in particular. In this paper, we propose a general partial face recognition approach that does not require face alignment by eye coordinates or any other fiducial points. We develop an alignment-free face representation method based on Multi-Keypoint Descriptors (MKD), where the descriptor size of a face is determined by the actual content of the image. In this way, any probe face image, holistic or partial, can be sparsely represented by a large dictionary of gallery descriptors. A new keypoint descriptor called Gabor Ternary Pattern (GTP) is also developed for robust and discriminative face recognition. Experimental results are reported on four public domain face databases (FRGCv2.0, AR, LFW, and PubFig) under both the open-set identification and verification scenarios. Comparisons with two leading commercial face recognition SDKs (PittPatt and FaceVACS) and two baseline algorithms (PCA+LDA and LBP) show that the proposed method, overall, is superior in recognizing both holistic and partial faces without requiring alignment.
INDEX TERMS
Face, Face recognition, Detectors, Image edge detection, Robustness, Lighting, Histograms,open-set identification, Partial face recognition, alignment free, keypoint descriptor, sparse representation
CITATION
Shengcai Liao, A. K. Jain, S. Z. Li, "Partial Face Recognition: Alignment-Free Approach", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 5, pp. 1193-1205, May 2013, doi:10.1109/TPAMI.2012.191
REFERENCES
 [1] G. Hua, M.-H. Yang, E. Learned-Miller, Y. Ma, M. Turk, D.J. Kriegman, and T.S. Huang, "Introduction to the Special Section on Real-World Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 1921-1924, Oct. 2011. [2] G.B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, "Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments," Technical Report 07-49, Univ. of Massachusetts, Amherst, http://vis-www.cs.umass.edulfw/, Oct. 2007. [3] FaceVACS Software Developer Kit, Cognitec Systems GmbH, http:/www.cognitec-systems.de, 2012. [4] PittPatt Software Developer Kit, Pittsburgh Pattern Recognition, Inc., http:/www.pittpatt.com, 2012. [5] "Police Use Facial Recognition Technology to Nab Rioters," http://www.msnbc.msn.com/id/44110353/ns/ technology_ and_science-tech_and_gadgets #.TkR_lnO4KsJ, 2012. [6] "Face Recognition Technology Fails to Find UK Rioters," http://www.newscientist.com/articlemg21128266.000-face-recognition-technology-fails-to-find-uk-rioters.html , 2012. [7] J. Wright, A.Y. Yang, A. Ganesh, S.S. Sastry, and Y. Ma, "Robust Face Recognition via Sparse Representation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210-227, Feb. 2009. [8] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, pp. 91-110, 2004. [9] S. Liao and A.K. Jain, "Partial Face Recognition: An Alignment Free Approach," Proc. IAPR/IEEE Int'l Joint Conf. Biometrics, Oct., 2011. [10] T.F. Cootes, C.J. Taylor, D. Cooper, and J. Graham, "Active Shape Models—Their Training and Application," Computer Vision and Image Understanding, vol. 61, no. 1, pp. 38-59, Jan. 1995. [11] T. Cootes, G. Edwards, and C. Taylor, "Active Appearance Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 681-685, June 2001. [12] A. Wagner, J. Wright, A. Ganesh, Z. Zhou, H. Mobahi, and Y. Ma, "Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 2, pp. 372-386, Feb. 2012. [13] S.Z. Li, X.W. Hou, and H.J. Zhang, "Learning Spatially Localized, Parts-Based Representation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001. [14] J. Kim, J. Choi, J. Yi, and M. Turk, "Effective Representation Using ICA for Face Recognition Robust to Local Distortion and Partial Occlusion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1977-1981, Dec. 2005. [15] H.J. Oh, K.M. Lee, and S.U. Lee, "Occlusion Invariant Face Recognition Using Selective Local Non-Negative Matrix Factorization Basis Images," Image and Vision Computing, vol. 26, no. 11, pp. 1515-1523, 2008. [16] K. Hotta, "Robust Face Recognition under Partial Occlusion Based on Support Vector Machine with Local Gaussian Summation Kernel," Image and Vision Computing, vol. 26, no. 11, pp. 1490-1498, 2008. [17] H. Jia and A. Martínez, "Support Vector Machines in Face Recognition with Occlusions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2009. [18] H. Ekenel and R. Stiefelhagen, "Why Is Facial Occlusion a Challenging Problem?" Proc. IAPR/IEEE Third Int'l Conf. Biometrics, 2009. [19] D. Beymer, "Face Recognition under Varying Pose," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 756-761, June 1994. [20] A. Pentland, B. Moghaddam, and T. Starner, "View-Based and Modular Eigenspaces for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 84-91, 1994. [21] T. Vetter and T. Poggio, "Linear Object Classes and Image Synthesis from a Single Example Image," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 733-741, July 1997. [22] D. Graham and N. Allison, "Face Recognition from Unfamiliar Views: Subspace Methods and Pose Dependency," Proc. Third Int'l Conf. Automatic Face and Gesture Recognition, pp. 348-353, 1998. [23] L. Wiskott, J.-M. Fellous, N. Krüger, and C. von der Malsburg, "Face Recognition by Elastic Bunch Graph Matching," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 775-779, July 1997. [24] V. Blanz, S. Romdhani, and T. Vetter, "Face Identification across Different Poses and Illumination with a 3D Morphable Model," Proc. Fifth Int'l Conf. Face and Gesture Recognition, pp. 202-207, 2002. [25] R. Gross, I. Matthews, and S. Baker, "Fisher Light-Fields for Face Recognition across Pose and Illumination," Proc. German Symp. Pattern Recognition, pp. 481-489, 2002. [26] B. Heisele, P. Ho, J. Wu, and T. Poggio, "Face Recognition: Component-Based versus Global Approaches," Computer Vision and Image Understanding, vol. 91, nos. 1/2, pp. 6-21, 2003. [27] J. Yang, S. Liao, and S.Z. Li, "Automatic Partial Face Alignment in NIR Video Sequences," Proc. IAPR/IEEE Third Int'l Conf. Biometrics, 2009. [28] D. Yi, S. Liao, Z. Lei, J. Sang, and S. Li, "Partial Face Matching between Near Infrared and Visual Images in MBGC Portal Challenge," Proc. IAPR/IEEE Third Int'l Conf. Biometrics, 2009. [29] K. Sato, S. Shah, and J. Aggarwal, "Partial Face Recognition Using Radial Basis Function Networks," Proc. IEEE Third Int'l Conf. Automatic Face and Gesture Recognition, pp. 288-293, 1998. [30] S. Gutta, V. Philomin, and M. Trajkovic, "An Investigation into the Use of Partial-Faces for Face Recognition," Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 33-38, 2002. [31] U. Park, A. Ross, and A. Jain, "Periocular Biometrics in the Visible Spectrum: A Feasibility Study," Proc. IEEE Third Int'l Conf. Biometrics: Theory, Applications, and Systems, 2009. [32] A. Martínez, "Recognizing Imprecisely Localized, Partially Occluded, and Expression Variant Faces from a Single Sample per Class," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp. 748-763, June 2002. [33] T. Ahonen, A. Hadid, and M. Pietikainen, "Face Description with Local Binary Patterns: Application to Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037-2041, Dec. 2006. [34] K. Pan, S. Liao, Z. Zhang, S. Li, and P. Zhang, "Part-Based Face Recognition Using Near Infrared Images," Proc. IEEE Int'l Workshop Object Tracking and Classification In and Beyond the Visible Spectrum, 2007. [35] R. Min, A. Hadid, and J.-L. Dugelay, "Improving the Recognition of Faces Occluded by Facial Accessories," Proc. IEEE Conf. Automatic Face and Gesture Recognition, 2011. [36] R. Brunelli and T. Poggio, "Face Recognition: Features versus Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 10, pp. 1042-1052, Oct. 1993. [37] C. Sanderson and K. Paliwal, "Polynomial Features for Robust Face Authentication," Proc. Int'l Conf. Image Processing, vol. 3, pp. 997-1000, 2002. [38] F. Cardinaux, C. Sanderson, and S. Marcel, "Comparison of MLP and GMM Classifiers for Face Verification on XM2VTS," Proc. Fourth Int'l Conf. Audio- and Video-Based Biometric Person Authentication, 2003. [39] S. Lucey and T. Chen, "A GMM Parts Based Face Representation for Improved Verification Through Relevance Adaptation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004. [40] F. Cardinaux, C. Sanderson, and S. Bengio, "User Authentication via Adapted Statistical Models of Face Images," IEEE Trans. Signal Processing, vol. 54, no. 1, pp. 361-373, Jan. 2006. [41] C. Chan and J. Kittler, "Sparse Representation of (Multiscale) Histograms for Face Recognition Robust to Registration and Illumination Problems," Proc. 17th IEEE Int'l Conf. Image Processing, pp. 2441-2444, 2010. [42] G. Csurka, C. Dance, L. Fan, J. Willamowski, and C. Bray, "Visual Categorization with Bags of Keypoints," Proc. ECCV Workshop Statistical Learning in Computer Vision, 2004. [43] D. Sun and Z. Qiu, "Bag-of-Words Vector Quantization Based Face Identification," Proc. Int'l Symp. Electronic Commerce and Security, vol. 2, pp. 29-33, 2009. [44] Z. Li, J. Imai, and M. Kaneko, "Robust Face Recognition Using Block-Based Bag of Words," Proc. 20th Int'l Conf. Pattern Recognition, Aug. 2010. [45] M. Bicego, A. Lagorio, E. Grosso, and M. Tistarelli, "On the Use of SIFT Features for Face Authentication," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshop, 2006. [46] J. Luo, Y. Ma, E. Takikawa, S. Lao, M. Kawade, and B. Lu, "Person-Specific SIFT Features for Face Recognition," Proc. IEEE Conf. Acoustics, Speech, and Signal Processing, 2007. [47] D. Kisku, A. Rattani, E. Grosso, and M. Tistarelli, "Face Identification by SIFT-Based Complete Graph Topology," Proc. IEEE Workshop Automatic Identification Advanced Technologies, 2007. [48] A. Mian, M. Bennamoun, and R. Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1927-1943, Nov. 2007. [49] L. Zhang, M. Yang, and X. Feng, "Sparse Representation or Collaborative Representation: Which Helps Face Recognition?" Proc. IEEE Int'l Conf. Computer Vision, 2011. [50] D. Yi, R. Liu, R. Chu, Z. Lei, and S.Z. Li, "Face Matching from Near Infrared to Visual Images," Proc. IAPR/IEEE Second Int'l Conf. Biometrics, Aug. 2007. [51] T. Lindeberg, "Feature Detection with Automatic Scale Selection," Int'l J. Computer Vision, vol. 30, no. 2, pp. 79-116, 1998. [52] J. Canny, "A Computational Approach to Edge Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp. 679-698, Nov. 1986. [53] K. Mikolajczyk, A. Zisserman, and C. Schmid, "Shape Recognition with Edge-Based Features," Proc. British Machine Vision Conf., 2003. [54] K. Mikolajczyk and C. Schmid, "Scale & Affine Invariant Interest Point Detectors," Int'l J. Computer Vision, vol. 60, no. 1, pp. 63-86, 2004. [55] K. Mikolajczyk and C. Schmid, "An Affine Invariant Interest Point Detector," Proc. European Conf. Computer Vision, 2002. [56] J. Matas, O. Chum, M. Urban, and T. Pajdla, "Robust Wide-Baseline Stereo from Maximally Stable Extremal Regions," Image and Vision Computing, vol. 22, no. 10, pp. 761-767, 2004. [57] "Scale & Affine Invariant Feature Detectors," http://www. robots.ox.ac.uk/vgg/research/ affine/det_eval_filesextract_ features2.tar.gz , 2012. [58] C. Liu and H. Wechsler, "Gabor Feature Based Classification Using the Enhanced Fisher Linear Discriminant Model for Face Recognition," IEEE Trans. Image Processing, vol. 11, no. 4, pp. 467-476, Apr. 2002. [59] X. Tan and B. Triggs, "Enhanced Local Texture Feature Sets for Face Recognition under Difficult Lighting Conditions," Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, 2007. [60] R. Tibshirani, "Regression Shrinkage and Selection via the Lasso," J. Royal Statistical Soc., Series B, vol. 58, pp. 267-288, 1994. [61] E. Candès, J. Romberg, and T. Tao, "Stable Signal Recovery from Incomplete and Inaccurate Measurements," Comm. Pure and Applied Math., vol. 59, no. 8, pp. 1207-1223, 2006. [62] H. Xu, C. Caramanis, and S. Mannor, "Sparse Algorithms Are Not Stable: A No-Free-Lunch Theorem," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 1, pp. 187-193, Jan. 2012. [63] D. Donoho and Y. Tsaig, "Fast Solution of $\ell_1$ -Norm Minimization Problems When the Solution May Be Sparse," IEEE Trans. Information Theory, vol. 54, no. 11, pp. 4789-4812, Nov. 2008. [64] D. Musser, "Introspective Sorting and Selection Algorithms," Software Practice and Experience, vol. 27, no. 8, pp. 983-993, 1997. [65] D. Beymer and T. Poggio, "Face Recognition from One Example View," Proc. Int'l Conf. Computer Vision, 1995. [66] P.J. Phillips, P.J. Flynn, T. Scruggs, K.W. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, "Overview of the Face Recognition Grand Challenge," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005. [67] A. Martinez and R. Benavente, "The AR Face Database," CVC technical report, 1998. [68] N. Kumar, A.C. Berg, P.N. Belhumeur, and S.K. Nayar, "Attribute and Simile Classifiers for Face Verification," Proc. IEEE Int'l Conf. Computer Vision, http://www.cs.columbia.edu/CAVE/databases pubfig/, Oct. 2009. [69] CSU Face Identification Evaluation System, http://www.cs. colostate.eduevalfacerec/, 2012. [70] Q. Yin, X. Tang, and J. Sun, "An Associate-Predict Model for Face Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011. [71] N. Pinto, J. DiCarlo, and D. Cox, "How Far Can You Get with a Modern Face Recognition Test Set Using Only Simple Features?" Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009. [72] P.J. Phillips, P. Grother, and R. Micheals, "Evaluation Methods in Face Recognition," Handbook of Face Recognition, second ed., S.Z. Li and A.K. Jain, eds., pp. 551-574, Springer, Sept. 2011.