This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Automatic Interpretation and Coding of Face Images Using Flexible Models
July 1997 (vol. 19 no. 7)
pp. 743-756

Abstract—Face images are difficult to interpret because they are highly variable. Sources of variability include individual appearance, 3D pose, facial expression , and lighting. We describe a compact parametrized model of facial appearance which takes into account all these sources of variability. The model represents both shape and gray-level appearance , and is created by performing a statistical analysis over a training set of face images. A robust multiresolution search algorithm is used to fit the model to faces in new images. This allows the main facial features to be located , and a set of shape , and gray-level appearance parameters to be recovered. A good approximation to a given face can be reconstructed using less than 100 of these parameters. This representation can be used for tasks such as image coding, person identification, 3D pose recovery, gender recognition , and expression recognition. Experimental results are presented for a database of 690 face images obtained under widely varying conditions of 3D pose, lighting , and facial expression. The system performs well on all the tasks listed above.

[1] T.N. Ahmed, "A Model Based Hand Gesture Recognition System," MSc Thesis, Dept. of Computer Science, Univ. of Manchester, 1994.
[2] M. Bichsel and A. Pentland, "Automatic Interpretation of Human Head Movements," MIT Media Laboratory, Vision and Modeling Group, Technical Report No. 186, 1993.
[3] L. Bookstein, "Principal Warps: Thin-Plate Splines and the Decomposition of Deformations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 6, pp. 567-585, June 1989.
[4] V. Bruce, Recognising Faces.London : LEA Publishers, 1991.
[5] R. Brunelli and T. Poggio, "Face Recognition: Features vs. Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 10, pp. 1,042-1,053, Oct. 1993.
[6] A.F. Clark and M. Kokuer, "A Model-Based Codec With Potential For Deaf Communication," Proc. 12th Int'l Conf. Pattern Recognition, vol. 3, pp. 195-197.Los Alamitos, Calif.: IEEE CS Press, 1994.
[7] T.F. Cootes, C.J. Taylor, A. Lanitis, D.H. Cooper, and J. Graham, "Building and Using Flexible Models Incorporating Grey-Level Information," Proc. Fourth Int'l Conf. Computer Vision, pp. 242-246.Los Alamitos, Calif.: IEEE CS Press, 1993.
[8] T.F. Cootes, A. Hill, C.J. Taylor, and J. Haslam, "The Use of Active Shape Models For Locating Structures in Medical Images," Image and Vision Computing, vol. 12, no. 6, pp. 355-366, 1994.
[9] T.F. Cootes, C.J. Taylor, and A. Lanitis, "Multi-Resolution Search Using Active Shape Models," Proc. 12th Int'l Conf. Pattern Recognition, vol. 1, pp. 610-612.Los Alamitos, Calif.: IEEE CS Press, 1994.
[10] T.F. Cootes, C.J. Taylor, D.H. Cooper, and J. Graham, "Active Shape Models—Their Training And Application," Computer Vision Graphics and Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.
[11] G.W. Cottrell and M.K. Fleming, "Categorisation of Faces Using Unsupervised Feature Extraction," Proc. Int'l Conf. Neural Networks,San Diego, vol. 2, pp. 65-70, 1990.
[12] G.W. Cottrell and J. Metcalfe, "EMPATH: Face, Gender and Emotion Recognition Using Holons," Advances in Neural Information Processing Systems, vol. 3, R.P. Lippman, J. Moody, and D.S. Touretzky, eds., pp. 564-571.San Mateo, Calif.: Kaufmann, 1991.
[13] I. Craw, D. Tock, and A. Bennett, "Finding Face Features," Proc. European Conf. Computer Vision, G. Sandini, ed. Springer-Verlag, 1992.
[14] I. Craw and P. Cameron, "Face Recognition by Computer," Proc. British Machine Vision Conference 1992, pp. 489-507, David Hogg and Roger Boyle, eds. Springer Verlag, 1992.
[15] P. Ekman and W. Friesen, Unmasking the Face, A Guide to Recognising Emotions From Facial Expressions. Prentice Hall, 1975.
[16] J.H. Friedman, "Requlised Discriminant Analysis," J. American Statistical Assn., vol. 84, no. 405, pp. 165-175, 1989.
[17] J. Fiser, I. Biederman, and E. Cooper, "To What Extent Can Matching Algorithms Based on Direct Outputs of Spatial Filters Account For Human Shape Recognition?," Filters and Recognition, Mar. 1993.
[18] A. Gee and R. Cipolla, "Estimating Gaze From a Single View of a Face," Proc. 12th Int'l Conf. Pattern Recognition, vol. 1, pp. 758-760.Los Alamitos, Calif.: IEEE CS Press, 1994.
[19] M. Kass, A. Witkin, D. Terzopoulos, "Snakes, Active Contour Models," First Int'l Conf. Computer Vision, pp. 259-268.Los Alamitos, Calif.: IEEE CS Press, 1987.
[20] M. Kirby and L. Sirovich,“Application of Karhunen-Loève procedure for the characterization of human faces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 103-108, Jan. 1990.
[21] M. Lades, J.C. Vorbruggen, J. Buhmann, J. Lange, C. von der Malsburg, R.P. Wurtz, and W. Konen, “Distortion Invariant Object Recognition in the Dynamic Link Architecture,” IEEE Trans. Computers, vol. 42, no. 3, pp. 300-311, Mar. 1993.
[22] A. Lanitis, C.J. Taylor, and T.F. Cootes, "Automatic Tracking, Coding and Reconstruction of Human Faces, Using Flexible Appearance Models," Electronics Letters, vol. 30, no. 19, pp. 1,578-1,579, 1994.
[23] A. Lanitis, C.J. Taylor, and T.F. Cootes, "Automatic Identification of Human Faces Using Flexible Appearance Models," Image and Vision Computing, vol. 13, no. 5, pp. 393-401, 1995.
[24] A. Lanitis, C.J. Taylor, and T.F. Cootes, "A Unified Approach to Coding and Interpreting Face Images," Int'l Conf. Computer Vision, 1995, pp. 368-373.
[25] A. Lanitis, C.J. Taylor, T.F. Cootes, and T. Ahmed, "Automatic Interpretation of Human Faces and Hand Gestures Using Flexible Templates," Proc. Int'l Workshop Face and Gesture Recognition, pp. 98-103, M. Bichsel, ed., Zurich, Switzerland, 1995.
[26] A. Lanitis, C.J. Taylor, T. Ahmed, and T.F. Cootes, "Classifying Variable Objects Using a Flexible Shape Model," Proc. Fifth Int'l Conf. Image Processing and its Applications, pp. 70-74,Edinburg, UK, 1995.
[27] H. Li, P. Roivainen, and R. Forchheimer, "3D Motion Estimation in Model-Based Facial Image Coding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 545-555, June 1993.
[28] B.F.J. Manly, Multivariate Statistical Methods, A Primer. Chapman and Hall, 1986.
[29] Y. Moses, Y. Adini, and S. Ulman, "Face Recognition: The Problem of Compensating For Changes in Illumination Direction," Proc. European Conf. Computer Vision, vol. 1, pp. 286-296, J. Eklundh, ed. Springer Verlag, 1994.
[30] A. Pentland, B. Moghaddam, T. Starner, O. Oliyide, and M. Turk, "View-Based and Modular Eigenspaces for Face Recognition," M.I.T Media Laboratory, Perceptual Computing Section, Technical Report No. 245, 1994.
[31] M. Rydfalk, "CANDICE, A Parametrized Face," Technical Report, Linkoping Univ., Dept. of Electrical Eng., S-581 83 Linkoping, Sweden, 1987.
[32] H.A. Sackeim, R. Gur, and M. Saucy, "Emotions Are Expressed More Intensively on The Left Side of The Face," Science, vol. 202, pp. 434-435, 1978.
[33] D. Terzopoulos and K. Waters, "Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 569-579, 1993.
[34] D. Tock and I. Craw, "Blink Rate Monitoring For a Driver Awareness System," Proc. British Machine Vision Conf. 1992, pp. 518-527, D. Hogg and R. Boyle, eds. Springer Verlag, 1992.
[35] A. Tsukamoto, C. Lee, and S. Tsuji, "Detection and Pose Estimation of Human Faces with Synthesized Image Models," Proc. 12th Int'l Conf. Pattern Recognition, vol. 1, pp. 754-757.Los Alamitos, Calif.: IEEE CS Press, 1994.
[36] M. Turk and A. Pentland, "Eigenfaces for Recognition," J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
[37] J.B. Waite and W.J. Welsh, "An Application of Active Contour Models to Head Boundary Location," Proc. First British Machine Vision Conference, pp. 407-412, 1990.
[38] Y. Yacoob and L. Davis, "Recognising Facial Expressions by Spatio-Temporal Analysis," Proc. 12th Int'l Conf. Pattern Recognition, vol. 1, pp. 747-749.Los Alamitos, Calif.: IEEE CS Press, 1994.
[39] A.L. Yuille, D.S. Cohen, and P.W. Hallinan, "Feature Extraction from Faces Using Deformable Templates," Proc. Computer Vision and Pattern Recognition,San Diego, June 1989.

Index Terms:
Face recognition, expression recognition, pose recovery, coding-reconstruction, facial feature location, deformable templates.
Citation:
Andreas Lanitis, Chris J. Taylor, Timothy F. Cootes, "Automatic Interpretation and Coding of Face Images Using Flexible Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 743-756, July 1997, doi:10.1109/34.598231
Usage of this product signifies your acceptance of the Terms of Use.