CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2009 vol.31 Issue No.11 - November

Subscribe

Issue No.11 - November (2009 vol.31)

pp: 1968-1984

Yang Wang , Carnegie Mellon University, Pittsburgh

Lei Zhang , Stony Brook University, Stony Brook

Zicheng Liu , Microsoft Research, Redmond

Gang Hua , Microsoft Live Labs Research, Redmond

Zhen Wen , IBM T.J. Watson Research Center, Hawthorne

Zhengyou Zhang , Microsoft Research, Redmond

Dimitris Samaras , Stony Brook University, Stony Brook

ABSTRACT

In this paper, we present a new method to modify the appearance of a face image by manipulating the illumination condition, when the face geometry and albedo information is unknown. This problem is particularly difficult when there is only a single image of the subject available. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using a spherical harmonic representation. Moreover, morphable models are statistical ensembles of facial properties such as shape and texture. In this paper, we integrate spherical harmonics into the morphable model framework by proposing a 3D spherical harmonic basis morphable model (SHBMM). The proposed method can represent a face under arbitrary unknown lighting and pose simply by three low-dimensional vectors, i.e., shape parameters, spherical harmonic basis parameters, and illumination coefficients, which are called the SHBMM parameters. However, when the image was taken under an extreme lighting condition, the approximation error can be large, thus making it difficult to recover albedo information. In order to address this problem, we propose a subregion-based framework that uses a Markov random field to model the statistical distribution and spatial coherence of face texture, which makes our approach not only robust to extreme lighting conditions, but also insensitive to partial occlusions. The performance of our framework is demonstrated through various experimental results, including the improved rates for face recognition under extreme lighting conditions.

INDEX TERMS

Face synthesis and recognition, Markov random field, 3D spherical harmonic basis morphable model, vision for graphics.

CITATION

Yang Wang, Lei Zhang, Zicheng Liu, Gang Hua, Zhen Wen, Zhengyou Zhang, Dimitris Samaras, "Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.31, no. 11, pp. 1968-1984, November 2009, doi:10.1109/TPAMI.2008.244REFERENCES