This Article 
 Bibliographic References 
 Add to: 
Recognizing Human Facial Expressions From Long Image Sequences Using Optical Flow
June 1996 (vol. 18 no. 6)
pp. 636-642

Abstract—An approach to the analysis and representation of facial dynamics for recognition of facial expressions from image sequences is presented. The algorithms utilize optical flow computation to identify the direction of rigid and nonrigid motions that are caused by human facial expressions. A mid-level symbolic representation motivated by psychological considerations is developed. Recognition of six facial expressions, as well as eye blinking, is demonstrated on a large set of image sequences.

[1] M. Abdel-Mottaleb, R. Chellappa, and A. Rosenfeld, "Binocular Motion Stereo Using MAP Estimation," IEEE Conf. Computer Vision and Pattern Recognition, pp. 321-327, 1993.
[2] J.N. Bassili, "Emotion Recognition: The Role of Facial Movement and the Relative Importance of Upper and Lower Areas of the Face," J. Personality and Social Psychology, vol. 37, pp. 2,049-2,059, 1979.
[3] D. Beymer, A. Shashua, and T. Poggio, "Example Based Image Analysis and Synthesis," M.I.T. A.I. Memo No. 1431, 1993.
[4] M.J. Black and Y. Yacoob, "Tracking and Recognizing Rigid and Non-Rigid Facial Motions Using Local Parametric Model of Image Motion," Proc. Int'l Conf. Computer Vision, pp. 374-381,Cambridge, Mass., 1995.
[5] R. Brunelli and T. Poggio, "Face Recognition: Features vs. Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 10, pp. 1,042-1,053, Oct. 1993.
[6] R. Chellappa, S.A. Sirohey, C.L. Wilson, and C.S. Barnes, "Human and Machine Recognition of Faces: A Survey," Center for Automation Research, Univ. of Maryland Technical Report CAR-TR-731, Aug. 1994.
[7] P. Ekman and W. Friesen, Unmasking the Face. Prentice Hall, 1975.
[8] P. Ekman and W. Friesen, The Facial Action Coding System.San Francisco: Consulting Psychologists Press, 1978.
[9] I. Essa and A. Pentland, "A Vision System for Observing and Extracting Facial Action Parameters," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994.
[10] I.A. Essa, "Analysis, Interpretation, and Synthesis of Facial Expressions," M.I.T. Media Laboratory, Perceptual Computing Group Report No. 303, 1994.
[11] H. Li, P. Roivainen, and R. Forchheimer, "3D Motion Estimation in Model-Based Facial Image Coding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 545-555, June 1993.
[12] K. Mase, "Recognition of Facial Expression from Optical Flow," IEICE Trans., vol. E74, pp. 3,474-3,483, 1991.
[13] K. Matsuno, C. Lee, and S. Tsuji, "Recognition of Human Facial Expressions Without Feature Extraction," Proc. ECCV, pp. 513-520, 1994.
[14] M. Rosenblum, Y. Yacoob, and L.S. Davis, "Human Emotion Recognition from Motion Using a Radial Basis Function Network Architecture," IEEE Workshop Motion of Non-Rigid and Articulated Objects,Austin, Texas, pp. 43-49, Nov. 1994.
[15] D. Terzopoulos and K. Waters, "Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 569-579, 1993.
[16] Y. Yacoob and L.S. Davis, "Labeling of Human Face Components from Range Data," CVGIP: Image Understanding, vol. 60, no. 2, pp. 168-178, 1994.
[17] Handbook of Research on Face Processing, A.W. Young and H.D. Ellis, eds. Elsevier Science Publishers, 1989.
[18] A.L. Yuille, D.S. Cohen, and P.W. Hallinan, "Feature Extraction from Faces Using Deformable Templates," Proc. Computer Vision and Pattern Recognition,San Diego, June 1989.

Index Terms:
Face expression recognition, non-rigis motion analysis, optical flow, tracking.
Yaser Yacoob, Larry S. Davis, "Recognizing Human Facial Expressions From Long Image Sequences Using Optical Flow," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 6, pp. 636-642, June 1996, doi:10.1109/34.506414
Usage of this product signifies your acceptance of the Terms of Use.