This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Face Recognition from a Single Training Image under Arbitrary Unknown Lighting Using Spherical Harmonics
March 2006 (vol. 28 no. 3)
pp. 351-363
In this paper, we propose two novel methods for face recognition under arbitrary unknown lighting by using spherical harmonics illumination representation, which require only one training image per subject and no 3D shape information. Our methods are based on the recent result which demonstrated that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace. We provide two methods to estimate the spherical harmonic basis images spanning this space from just one image. Our first method builds the statistical model based on a collection of 2D basis images. We demonstrate that, by using the learned statistics, we can estimate the spherical harmonic basis images from just one image taken under arbitrary illumination conditions if there is no pose variation. Compared to the first method, the second method builds the statistical models directly in 3D spaces by combining the spherical harmonic illumination representation and a 3D morphable model of human faces to recover basis images from images across both poses and illuminations. After estimating the basis images, we use the same recognition scheme for both methods: we recognize the face for which there exists a weighted combination of basis images that is the closest to the test face image. We provide a series of experiments that achieve high recognition rates, under a wide range of illumination conditions, including multiple sources of illumination. Our methods achieve comparable levels of accuracy with methods that have much more onerous training data requirements. Comparison of the two methods is also provided.

[1] R. Chellappa, C.L. Wilson, and S. Sirohey, “Human and Machine Recognition of Faces: A Survey,” Proc. IEEE, vol. 83, no. 5, pp. 705-740, 1995.
[2] A.P. Pentland, “Looking at People: Sensing for Ubiquitous and Wearable Computing,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 107-119, Jan. 2000.
[3] W. Zhao, R. Chellappa, A. Rosenfeld, and P.J. Phillips, “Face Recognition: A Literature Survey,” Technical Report CAR-TR-948, Univ. of Maryland, CfAR, 2000.
[4] R. Gross, S. Baker, I. Matthews, and T. Kanade, “Face Recognition across Pose and Illumination,” Handbook of Face Recognition, Springer-Verlag, 2004.
[5] A. Shashua and R.T. Riklin, “The Quotient Image: Class Based Re-Rendering and Recognition with Varying Illuminations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 129-139, Feb. 2001.
[6] M. Turk and A.P. Pentland, “Eigenfaces for Recognition,” J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-96, 1991.
[7] H. Murase and S.K. Nayar, “Visual Learning and Recognition of 3-D Objects from Appearance,” Int'l J. Computer Vision, vol. 14, no. 1, pp. 5-24, 1995.
[8] A.S. Georghiades and P.N. Belhumeur, and D.J. Kriegman, “From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001.
[9] S.K. Zhou, R. Chellappa, and D.W. Jacobs, “Characterization of Human Faces under Illumination Variations Using Rank, Integrability, and Symmetry Constraints,” Proc. European Conf. Computer Vision, 2004.
[10] S. Zhou and R. Chellappa, “Illuminating Light Field: Image-Based Face Recognition across Illuminations and Poses,” Proc. Sixth IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2004.
[11] V. Blanz and T. Vetter, “A Morphable Model for the Synthesis of 3D-Faces,” Proc. SIGGRAPH, 1999.
[12] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[13] V. Blanz, K. Scherbaum, T. Vetter, and H. Seidel, “Exchanging Faces in Images,” Proc. EuroGraphics, 2004.
[14] S. Romdhani, V. Blanz, and T. Vetter, “Face Identification by Fitting a 3D Morphable Model Using Linear Shape and Texture Error Functions,” Proc. European Conf. Computer Vision, 2002.
[15] R. Basri and D.W. Jacobs, “Lambertian Reflectance and Linear Subspaces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[16] R. Ramamoorthi, “Analytic PCA Construction for Theoretical Analysis of Lighting Variability in Images of a Lambertian Object,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 10, Oct. 2002.
[17] P.W. Hallinan, “A Low-Dimensional Representation of Human Faces for Arbitrary Lighting Conditions,” Proc. Computer Vision and Pattern Recognition, pp. 995-999, 1994.
[18] P.N. Belhumeur and D.J. Kriegman, “What Is the Set of Images of an Object under All Possible Illumination Conditions,” Int'l J. Computer Vision, vol. 28, no. 3, pp. 245-260, 1998.
[19] Y. Adini, Y. Moses, and S. Ullman, “Face Recognition: The Problem of Compensating for Changes in Illumination Direction,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 721-732, July 1997.
[20] A. Shashua, “Illumination and View Position in 3D Visual Recognition,” Proc. Advances in Neural Information Processing Systems 4, 1991.
[21] S.Z. Li, J. Yan, X. Hou, Z. Li, and H. Zhang, “Learning Low Dimensional Invariant Signature of 3-D Object under Varying View and Illumination from 2-D Appearances,” Proc. Int'l Conf. Computer Vision, pp. 635-640, 2001.
[22] H. Wang, S.Z. Li, Y. Wang, and W. Zhang, “Illumination Modeling and Normalization for Face Recognition,” Proc. ICCV Workshop on Analysis and Modeling of Faces and Gestures, 2003.
[23] W.Y. Zhao and R. Chellappa, “Illumination-Insensitive Face Recognition Using Symmetric Shape-from-Shading,” Proc. Computer Vision and Pattern Recognition, vol. 1, pp. 286-293, 2000.
[24] L. Zhang and D. Samaras, “Face Recognition under Variable Lighting Using Harmonic Image Exemplars,” Proc. Computer Vision and Pattern Recognition, vol. 1, pp. 19-25, 2003.
[25] L. Zhang and D. Samaras, “Pose Invariant Face Recognition under Arbitrary Unknown Lighting Using Spherical Harmonics,” Biometric Authentication, vol. 1, pp. 10-23, 2004.
[26] P.N. Belhumeur, J.P. Hespanha, and D.J. Kriegman, “Eigenfaces versus Fisherfaces: Recognition Using Class-Specific Linear Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[27] T. Sim and T. Kanade, “Combining Models and Exemplars for Face Recognition: An Illuminating Example,” Proc. CVPR Workshop Models versus Exemplars in Computer Vision, 2001.
[28] T. Sim, S. Baker, and M. Bsat, “The CMU Pose, Illumination, and Expression (PIE) Database of Human Faces,” Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 46-51, 2002.
[29] R. Zhang, P.S. Tsai, J.E. Cryer, and M. Shah, “Shape from Shading: A Survey,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 8, pp. 690-706, Aug. 1999.
[30] J. Foley and A. van Dam, “Fundamentals of Interactive Computer Graphics,” The Systems Programming Series, Addison-Wesley, 1984.
[31] B.W. Hwang, V. Blanz, T. Vetter, and S.W. Lee, “Face Reconstruction from a Small Number of Feature Points,” Proc. Int'l Conf. Pattern Recognition, vol. 2, pp. 842-845, 2000.
[32] W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing. Cambridge Univ. Press, 1992.
[33] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[34] K.C. Lee, J. Ho, and D.J. Kriegman, “Nine Points of Light: Acquiring Subspaces for Face Recognition under Variable Lighting,” Proc. Computer Vision and Pattern Recognition, vol. 1, pp. 519-526, 2001.
[35] H.F. Chen, P.N. Belhumeur, and D.W. Jacobs, “In Search of Illumination Invariants,” Proc. Computer Vision and Pattern Recognition, pp. 1254-1261, 2000.
[36] H. Wu, T. Shimada, Q. Chen, K. Chihara, and T. Shioyama, “A Robust Algorithm for 3D Head Pose Estimation,” Proc. Int'l Conf. Pattern Recognition, 1998.
[37] J. Sherrah and S. Gong, “Fusion of 2D Face Alignment and 3D Head Pose Estimation for Robust and Real-Time Performance,” Proc. IEEE Int'l Workshop Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, 1999.
[38] V. Kruger and G. Sommer, “Gabor Wavelet Networks for Efficient Head Pose Estimation,” Image and Vision Computing, vol. 20, nos. 9-10, pp. 665-672, 2002.
[39] J.R. Bergen and R. Hingorani, “Hierarchical Motion-Based Frame Rate Conversion,” technical report, David Sarnoff Research Center, Princeton, N.J., 1990.

Index Terms:
Index Terms- Face recognition, spherical harmonics illumination representation, 3D face morphable models, illumination invariance.
Citation:
Lei Zhang, Dimitris Samaras, "Face Recognition from a Single Training Image under Arbitrary Unknown Lighting Using Spherical Harmonics," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 351-363, March 2006, doi:10.1109/TPAMI.2006.53
Usage of this product signifies your acceptance of the Terms of Use.