This Article 
 Bibliographic References 
 Add to: 
Reflectance from Images: A Model-Based Approach for Human Faces
May/June 2005 (vol. 11 no. 3)
pp. 296-305
In this paper, we present an image-based framework that acquires the reflectance properties of a human face. A range scan of the face is not required. Based on a morphable face model, the system estimates the 3D shape and establishes point-to-point correspondence across images taken from different viewpoints and across different individuals' faces. This provides a common parameterization of all reconstructed surfaces that can be used to compare and transfer BRDF data between different faces. Shape estimation from images compensates deformations of the face during the measurement process, such as facial expressions. In the common parameterization, regions of homogeneous materials on the face surface can be defined a priori. We apply analytical BRDF models to express the reflectance properties of each region and we estimate their parameters in a least-squares fit from the image data. For each of the surface points, the diffuse component of the BRDF is locally refined, which provides high detail. We present results for multiple analytical BRDF models, rendered at novel orientations and lighting conditions.

[1] Y. Sato, M.D. Wheeler, and K. Ikeuchi, “Object Shape and Reflectance Modeling from Observation,” Proc. SIGGRAPH '97, pp. 379-388, Aug. 1997.
[2] R. Lu, J. Koenderink, and A. Kappers, “Optical Properties (Bidirectional Reflectance Distribution Functions) of Velvet,” Applied Optics, vol. 37, no. 25, pp. 5974-5984, Sept. 1998.
[3] Y. Yu and J. Malik, “Recovering Photometric Properties of Architectural Scenes from Photographs,” Proc. SIGGRAPH '98, pp. 207-218, July 1998.
[4] W. Matusik, H. Pfister, M. Brand, and L. McMillan, “A Data-Driven Reflectance Model,” ACM SIGGRAPH, vol. 22, no. 3, pp. 759-769, July 2003.
[5] W. Matusik, H. Pfister, M. Brand, and L. McMillan, “Efficient Isotropic BRDF Measurement,” Proc. Eurographics Symp. Rendering: 14th Eurographics Workshop Rendering, pp. 241-248, June 2003.
[6] Y. Yu, P. Debevec, J. Malik, and T. Hawkins, “Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs,” Proc. SIGGRAPH, pp. 215-224, Aug. 1999.
[7] S. Gibson, T. Howard, and R. Hubbold, “Flexible Image-Based Photometric Reconstruction Using Virtual Light Sources,” Computer Graphics Forum, vol. 20, no. 3, 2001.
[8] S. Boivin and A. Gagalowicz, “Image-Based Rendering of Diffuse, Specular and Glossy Surfaces from a Single Image,” Proc. SIGGRAPH 2001, Computer Graphics Proc., Ann. Conf. Series, pp. 107-116, Aug. 2001.
[9] R. Ramamoorthi and P. Hanrahan, “A Signal-Processing Framework for Inverse Rendering,” Proc. SIGGRAPH 2001, Computer Graphics Proc., Ann. Conf. Series, pp. 117-128, Aug. 2001.
[10] K. Nishino, Z. Zhang, and K. Ikeuchi, “Determining Reflectance Parameters and Illumination Distribution from a Sparse Set of Images for View-Dependent Image Synthesis,” Proc. Eighth IEEE Intl Conf. Computer Vision (ICCV '01), pp. 599-606, July 2001.
[11] H.P.A. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H.-P. Seidel, “Image-Based Reconstruction of Spatial Appearance and Geometric Detail,” ACM Trans. Graphics, vol. 27, no. 2, Apr. 2003.
[12] S. Marschner, S. Westin, E. Lafortune, K. Torrance, and D. Greenberg, “Image-Based BRDF Measurement Including Human Skin,” Proc. 10th Eurographics Workshop Rendering, pp. 131-144, June 1999.
[13] E.P.F. Lafortune, S.-C. Foo, K.E. Torrance, and D.P. Greenberg, “Non-Linear Approximation of Reflectance Functions,” Proc. 24th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 117-126, 199.7
[14] S. Marschner, B. Guenter, and S. Raghupathy, “Modeling and Rendering for Realistic Facial Animation,” Proc. 11th Eurographics Workshop Rendering, pp. 231-242, June 2000.
[15] P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, “Acquiring the Reflectance Field of a Human Face,” Proc. 27th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 145-156, 2000.
[16] K. Torrance and E. Sparrow, “Theory for Off-Specular Reflection from Roughened Surfaces,” J. Optical Soc. Am., vol. 57, no. 9, pp. 1105-1114, 1967.
[17] B.T. Phong, “Illumination for Computer Generated Pictures,” Comm. ACM, vol. 18, no. 6, pp. 311-317, June 1975.
[18] S. Paris, F.X. Sillion, and L. Quan, “Lightweight Face Relighting,” Proc. 11th Pacific Conf. Computer Graphics and Applications, p. 41, 2003.
[19] B. Moghaddam, J. Lee, H. Pfister, and R. Machiraju, “Model-Based 3D Face Capture with Shape-from-Silhouettes,” AMFG '03: Proc. IEEE Int'l Workshop Analysis and Modeling of Faces and Gestures, p. 20, 2003.
[20] J. Lee, B. Moghaddam, H. Pfister, and R. Machiraju, “Finding Optimal Views for 3D Face Shape Modeling,” Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition (FG), May 2004.
[21] V. Blanz and T. Vetter, “A Morphable Model for the Synthesis of 3D Faces,” Computer Graphics Proc. SIGGRAPH '99, pp. 187-194, 1999.
[22] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, 2003.
[23] R.L. Carceroni and K.N. Kutulakos, “Multi-View Scene Capture by Surfel Sampling: From Video Streams to Non-Rigid 3D Motion, Shape and Reflectance,” Int'l J. Computer Vision, vol. 49, nos. 2-3, pp. 175-214, 2002.
[24] A.S. Georghiades, “Recovering 3-D Shape and Reflectance from a Small Number of Photographs,” Rendering Techniques 2003: Proc. 14th Eurographics Workshop Rendering, pp. 230-240, June 2003.
[25] N. Tsumura, N. Ojima, K. Sato, M. Shiraishi, H. Shimizu, H. Nabeshima, S. Akazaki, K. Hori, and Y. Miyake, “Image-Based Skin Color and Texture Analysis/Synthesis by Extracting Hemoglobin and Melanin Information in the Skin,” ACM Trans. Graphics, vol. 22, no. 3, pp. 770-779, 2003.
[26] A. Krishnaswamy and G.V.G. Baranoski, “A Biophysically-Based Spectral Model of Light Interaction with Human Skin,” Proc. Eurographics, 2004.
[27] H.W. Jensen, S. Marschner, M. Levoy, and P. Hanrahan, “A Practical Model for Subsurface Light Transport,” Proc. SIGGRAPH, pp. 511-518, Aug. 2001.
[28] M. Goesele, H.P.A. Lensch, W. Heidrich, and H.-P. Seidel, “Building a Photo Studio for Measurement Purposes,” Proc. Vision, Modeling, and Visualization (VMV-00), pp. 231-238, 2000.
[29] J.-Y. Bouguet, “Camera Calibration Toolbox for Matlab,” /, 2005.
[30] T. Vetter and T. Poggio, “Linear Object Classes and Image Synthesis from a Single Example Image,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 733-742, July 1997.
[31] V. Blanz, C. Basso, T. Poggio, and T. Vetter, “Reanimating Faces in Images and Video,” Computer Graphics Forum, P. Brunet and D. Fellner, eds., vol. 22, no. 3, pp. 641-650, 2003.
[32] W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Numerical Recipes in C: The Art of Scientific Computing, second ed. Cambridge Univ. Press, 1994.
[33] P. Debevec, “Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography,” Proc. 25th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 189-198, July 1998.
[34] R.L. Cook and K.E. Torrance, “A Reflectance Model for Computer Graphics,” ACM Trans. Graphics, vol. 1, no. 1, pp. 7-24, 1982.
[35] G. Ward Larson, “Measuring and Modeling Anisotropic Reflection,” Proc. SIGGRAPH, pp. 265-272, July 1992.
[36] P. Hanrahan and W. Krueger, “Reflection from Layered Surfaces Due to Subsurface Scattering,” Proc. SIGGRAPH '93, Computer Graphics Proc., Ann. Conf. Series, pp. 165-174, Aug. 1993.
[37] J. Koenderink and S. Pont, “The Secret of Velvety Skin,” Machine Vision Applications, vol. 14, no. 4, pp. 260-268, 2003.
[38] M. Goesele, H.P.A. Lensch, J. Lang, C. Fuchs, and H.-P. Seidel, “Disco: Acquisition of Translucent Objects,” ACM Trans. Graphics, vol. 23, no. 3, pp. 835-844, 2004.

Index Terms:
Color, shading, shadowing, texture, reflectance.
Martin Fuchs, Volker Blanz, Hendrik Lensch, Hans-Peter Seidel, "Reflectance from Images: A Model-Based Approach for Human Faces," IEEE Transactions on Visualization and Computer Graphics, vol. 11, no. 3, pp. 296-305, May-June 2005, doi:10.1109/TVCG.2005.47
Usage of this product signifies your acceptance of the Terms of Use.