Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. (2004)
Washington, D.C., USA
June 27, 2004 to July 2, 2004
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/CVPR.2004.39
Mingli Song , Zhejiang University
Jiajun Bu , Zhejiang University
Chun Chen , Zhejiang University
Nan Li , Zhejiang University
Emotion recognition is one of the latest challenges in intelligent human/computer communication. Most of previous work on emotion recognition focused on extracting emotions from visual or audio information separately. A novel approach is presented in this paper, including both visual and audio from video clips, to recognize the human emotion. The Facial Animation Parameters (FAPs) compliant facial feature tracking based on active appearance model is performed on the video to generate two vector stream which represent the expression feature and the visual speech one. Combined with the visual vectors, the audio vector is extracted in terms of low level features. Then, a tripled Hidden Markov Model is introduced to perform the recognition which allows the state asynchrony of the audio and visual observation sequences while preserving their natural correlation over time. The experimental results show that this approach outperforms only using visual or audio separately.
M. Song, J. Bu, N. Li and C. Chen, "Audio-Visual Based Emotion Recognition — A New Approach," Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004.(CVPR), Washington, D.C., USA, 2004, pp. 1020-1025.