This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Recovering Facial Shape Using a Statistical Model of Surface Normal Direction
December 2006 (vol. 28 no. 12)
pp. 1914-1930
In this paper, we show how a statistical model of facial shape can be embedded within a shape-from-shading algorithm. We describe how facial shape can be captured using a statistical model of variations in surface normal direction. To construct this model, we make use of the azimuthal equidistant projection to map the distribution of surface normals from the polar representation on a unit sphere to Cartesian points on a local tangent plane. The distribution of surface normal directions is captured using the covariance matrix for the projected point positions. The eigenvectors of the covariance matrix define the modes of shape-variation in the fields of transformed surface normals. We show how this model can be trained using surface normal data acquired from range images and how to fit the model to intensity images of faces using constraints on the surface normal direction provided by Lambert's law. We demonstrate that the combination of a global statistical constraint and local irradiance constraint yields an efficient and accurate approach to facial shape recovery and is capable of recovering fine local surface details. We assess the accuracy of the technique on a variety of images with ground truth and real-world images.

[1] “USF HumanID 3D Face Database,” Courtesy of Sudeep Sarkar, Univ. of South Florida, Tampa, 2004.
[2] J.J. Atick, P.A. Griffin, and A.N. Redlich, “Statistical Approach to SFS: Reconstruction of 3D Face Surfaces from Single 2D Images,” Neural Computation, vol. 8, no. 6, pp. 1321-1340, 1996.
[3] V. Blanz, K. Scherbaum, T. Vetter, and H.P. Seidel, “Exchanging Faces in Images,” Proc. EUROGRAPHICS, pp. 669-676, 2004.
[4] V. Blanz and T. Vetter, “A Morphable Model for the Synthesis of 3D Faces,” Proc. Computer Graphics SIGGRAPH, pp. 187-194, 1999.
[5] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[6] V. Bruce, A. Coombes, and R. Richards, “Describing the Shapes of Faces Using Surface Primitives,” Image and Vision Computing, vol. 11, no. 6, pp. 353-363, 1993.
[7] M. Castelán and E.R. Hancock, “Acquiring Height Maps of Faces from a Single Image,” Proc. Int'l Symp. 3D Data Processing Visualization and Transmission, pp. 183-190, 2004.
[8] T.F. Cootes, G.J. Edwards, and C.J. Taylor, “Active Appearance Models,” Proc. European Conf. Computer Vision, pp. 484-498, 1998.
[9] T.F. Cootes, C.J. Taylor, D. Cooper, and J. Graham, “Training Models of Shape from Sets of Examples,” Proc. British Machine Vision Conf., pp. 9-18, 1992.
[10] N.P. Costen, T.F. Cootes, G.J. Edwards, and C.J. Taylor, “Automatic Extraction of the Face Identity-Subspace,” Image and Vision Computing, vol. 20, pp. 319-329, 2002.
[11] M.P. Do Carmo, Differential Geometry of Curves and Surfaces. Prentice-Hall, 1976.
[12] R. Dovgard and R. Basri, “Statistical Symmetric Shape from Shading for 3D Structure Recovery of Faces,” Proc. European Conf. Computer Vision, vol. 2, pp. 99-113, 2004.
[13] R.A. Fisher, “Dispersion on a Sphere,” Proc. Royal Soc., Ser. A, vol. 217, pp. 295-305, 1953.
[14] R.T. Frankot and R. Chellappa, “A Method for Enforcing Integrability in Shape from Shading Algorithms,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 4, pp. 439-451, Apr. 1988.
[15] A. Georghiades, P. Belhumeur, and D. Kriegman, “From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, pp. June 2001.
[16] R.L. Gregory, “Knowledge in Perception and Illusion,” Philosophical Trans. Royal Soc. London B, vol. 352, pp. 1121-1128, 1997.
[17] T. Heap and D. Hogg, “Extending the Point Distribution Model Using Polar Coordinates,” Image and Vision Computing, vol. 14, pp.589-599, 1996.
[18] G.W. Hill, “Algorithm 571: Statistics for von Mises' and Fisher's Distributions of Directions: $I_{1}(x)/I_{0}(x)$ , $I_{1.5}(x)/I_{0.5}(x)$ and Their Inverses,” ACM Trans. Math. Software, vol. 7, no. 2, pp. 233-238, 1981.
[19] H. Hill and V. Bruce, “Effects of Lighting on the Perception of Facial Surfaces,” J. Experimental Psychology: Human Perception and Performance, vol. 22, no. 4, pp. 986-1004, 1996.
[20] A. Johnston, H. Hill, and N. Carman, “Recognising Faces: Effects of Lighting Direction, Inversion, and Brightness Reversal,” Perception, vol. 21, pp. 365-375, 1992.
[21] A. Lanitis, C.J. Taylor, and T.F. Cootes, “Towards Automatic Simulation of Ageing Effects on Face Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 442-455, Apr. 2002.
[22] K.V. Mardia and P.E. Jupp, Directional Statistics. John Wiley and Sons Ltd., 2000.
[23] D. Marr, Vision. W.H. Freeman and Co., 1982.
[24] S.R. Marschner, B. Guenter, and S. Raghupathy, “Modeling and Rendering for Realistic Facial Animation,” Proc. 11th Eurographics Workshop Rendering, pp. 231-242, 2000.
[25] S. Marschner, S. Westin, E. Lafortune, K. Torrance, and D. Greenberg, “Reflectance Measurements of Human Skin,” Technical Report PCG-99-2, Program of Computer Graphics, Cornell Univ., 1999.
[26] I. Matthews, T.F. Cootes, and J.A. Bangham, “Extraction of Visual Features for Lipreading,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 2, pp. 198-213, Feb. 2002.
[27] D. Nandy and J. Ben-Arie, “Shape from Recognition: A Novel Approach for 3-D Face Shape Recovery,” IEEE Trans. Image Processing, vol. 10, no. 2, pp. 206-217, 2001.
[28] X. Pennec, “Probabilities and Statistics on Riemannian Manifolds: A Geometric Approach,” Technical Report RR-5093, INRIA, 2004.
[29] A.P. Pentland, “Finding the Illuminant Direction,” J. Optical Soc. Am., vol. 72, no. 4, pp. 448-455, 1982.
[30] E. Prados and O. Faugeras, “A Rigorous and Realistic Shape from Shading Method and Some of Its Applications,” Technical Report RR-5133, INRIA, 2004.
[31] A. Robles-Kelly and E.R. Hancock, “Estimating the Surface Radiance Function from Single Images,” Graphical Models, vol. 67, no. 6, pp. 518-548, 2005.
[32] D. Samaras and D. Metaxas, “Illumination Constraints in Deformable Models for Shape and Light Direction Estimation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 247-264, Feb. 2003.
[33] L. Sirovich, “Turbulence and the Dynamics of Coherent Structures,” Quartly Applied Math., vol. XLV, no. 3, pp. 561-590, 1987.
[34] J.P. Snyder, Map Projections—A Working Manual, USGS Professional Paper 1395. US Government Printing Office, 1987.
[35] N.F. Troje and H.H. Bülthoff, “Face Recognition under Varying Poses: The Role of Texture and Shape,” Vision Research, vol. 36, no. 12, pp. 1761-1771, 1996.
[36] M. Turk and A. Pentland, “Face Recognition Using Eigenfaces,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 586-591, 1991.
[37] M.J.C. van Gemert, S.L. Jacques, H.J.C.M. Sterenborg, and W.M. Star, “Skin Optics,” IEEE Trans. Biomedical Eng., vol. 36, no. 12, pp.1146-1154, 1989.
[38] P.L. Worthington and E.R. Hancock, “New Constraints on Data-Closeness and Needle Map Consistency for Shape-from-Shading,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1250-1267, Dec. 1999.
[39] P.L. Worthington and E.R. Hancock, “Surface Topography Using Shape-from-Shading,” Pattern Recognition, vol. 34, no. 4, pp. 823-840, 2001.
[40] J. Xiao, S. Baker, I. Matthews, and T. Kanade, “Real-Time Combined $2{\rm D} + 3{\rm D}$ Active Appearance Models,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 535-542, 2004.
[41] R. Zhang, P.S. Tsai, J.E. Cryer, and M. Shah, “Shape-from-Shading: A Survey,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 8, pp. 690-706, Aug. 1999.
[42] W.Y. Zhao and R. Chellappa, “Symmetric Shape-from-Shading Using Self-Ratio Image,” Int'l J. Computer Vision, vol. 45, pp. 55-75, 2001.

Index Terms:
Shape-from-shading, albedo estimation, directional statistics, illumination, face modeling.
Citation:
William A.P. Smith, Edwin R. Hancock, "Recovering Facial Shape Using a Statistical Model of Surface Normal Direction," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 1914-1930, Dec. 2006, doi:10.1109/TPAMI.2006.251
Usage of this product signifies your acceptance of the Terms of Use.