This Article 
 Bibliographic References 
 Add to: 
Analyzing Facial Expressions for Virtual Conferencing
September-October 1998 (vol. 18 no. 5)
pp. 70-78
We present a method for estimating 3D motion from 2D image sequences showing head and shoulder scenes typical of video telephone and teleconferencing applications. Our 3D model specifies the color and shape of the person in the video. Additionally, the model constrains facial motion and deformation to a set of facial expressions represented by the facial animation parameters (FAPs) defined by the MPEG-4 standard. Using this model, we obtain a description of both global and local 3D head motion as a function of the unknown facial parameters. Combining the 3D information with the optical flow constraint leads to a robust and linear algorithm that estimates the facial animation parameters from two successive frames with low computational complexity. Experimental results on synthetic and real data confirm the technique's applicability and show that image sequences of head and shoulder scenes can be encoded at bit rates below 0.6 Kbits.

1. D.E. Pearson, "Developments in Model-Based Video Coding," Proc. IEEE, Vol. 83, No. 6, June 1995, pp. 892-906.
2. W.J. Welsh, S. Searsby, and J.B. Waite, "Model-Based Image Coding," British Telecom Technology J., Vol. 8, No. 3, July 1990, pp. 94-106.
3. K. Aizawa and T.S. Huang, “Model-Based Image Coding: Advanced Video Coding Techniques for Very Low Bit-Rate Application,” Proc. IEEE, vol. 83, pp. 259-271, Aug. 1995.
4. I.S. Pandzic et al., "Towards Natural Communication in Networked Collaborative Virtual Environments," Proc. Framework for Immersive Environments (FIVE) 96, 1996, .
5. F.I. Parke, "Parameterized Models for Facial Animation," IEEE CG&A, Vol. 2, No. 9, 1982, pp. 61-68.
6. M. Rydfalk, Candide: A Parameterized Face, LiTH-ISY-I-0866, Image Coding Group, Linköping Univ., Linköping, Sweden, Oct., 1987.
7. G. Greiner and H.-P. Seidel, "Modeling with Triangular B-Splines," Proc. ACM/IEEE Solid Modeling Symp. 93, 1993, ACM Press, New York, pp. 211-220.
8. M. Hoch, G. Fleischmann, and B. Girod, "Modeling and Animation of Facial Expressions Based on B-Splines," Visual Computer, Vol. 11, 1994, pp. 87-95.
9. SNHC Systems Verification Model 4.0, ISO/IEC JTC1/SC29/WG11 N1666, Bristol, Great Britain, April 1997.
10. R. Tsai, "A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses," IEEE J. Robotics and Automation, vol. 3, no. 4, pp. 323-344, Aug. 1987.
11. R. Koch, "Dynamic 3D Scene Analysis through Synthesis Feedback Control," IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 15, No. 6, June 1993, pp. 556-568.
12. H. Li, P. Roivainen, and R. Forchheimer, "3D Motion Estimation in Model-Based Facial Image Coding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 545-555, June 1993.
13. J. Ostermann, "Object-Based Analysis-Synthesis Coding (OBACS) Based on the Source Model of Moving Flexible 3D Objects," IEEE Trans. Image Processing, Vol. 3, No. 5, Sept. 1994, pp. 705-711.
14. D. DeCarlo and D. Metaxas, “The Integration of Optical Flow and Deformable Models: Applications to Human Face Shape and Motion Estimation,” Proc. IEEE Computer Vision and Pattern Recognition (CVPR '96), pp. 231-238, 1996.
15. P. Eisert and B. Girod, "Model-Based Coding of Facial Image Sequences at Varying Illumination Conditions," Proc. 10th Image and Multidimensional Digital Signal Processing Workshop 98, IEEE Press, Piscataway, N.J., July 1998, pp. 119-122.

Index Terms:
model-based video coding, 3D head model, MPEG-4, facial expression analysis, virtual conferencing.
Peter Eisert, Bernd Girod, "Analyzing Facial Expressions for Virtual Conferencing," IEEE Computer Graphics and Applications, vol. 18, no. 5, pp. 70-78, Sept.-Oct. 1998, doi:10.1109/38.708562
Usage of this product signifies your acceptance of the Terms of Use.