Issue No. 04 - Oct.-Dec. (2016 vol. 7)
Michael Xuelin Huang , Department of Computing, Hong Kong Polytechnic University, Hong Kong
Grace Ngai , Department of Computing, Hong Kong Polytechnic University, Hong Kong
Kien A. Hua , School of Electrical Engineering and Computer Science, University of Central Florida, FL, Orlando
Stephen C.F. Chan , Department of Computing, Hong Kong Polytechnic University, Hong Kong
Hong Va Leong , Department of Computing, Hong Kong Polytechnic University, Hong Kong
This paper presents Personalized Affect Detection with Minimal Annotation (PADMA), a user-dependent approach for identifying affective states from spontaneous facial expressions without the need for expert annotation. The conventional approach relies on the use of key frames in recorded affect sequences and requires an expert observer to identify and annotate the frames. It is susceptible to user variability and accommodating individual differences is difficult. The alternative is a user-dependent approach, but it would be prohibitively expensive to collect and annotate data for each user. PADMA uses a novel Association-based Multiple Instance Learning (AMIL) method, which learns a personal facial affect model through expression frequency analysis, and does not need expert input or frame-based annotation. PADMA involves a training/calibration phase in which the user watches short video segments and reports the affect that best describes his/her overall feeling throughout the segment. The most indicative facial gestures are identified and extracted from the facial response video, and the association between gesture and affect labels is determined by the distribution of the gesture over all reported affects. Hence both the geometric deformation and distribution of key facial gestures are specially adapted for each user. We show results that demonstrate the feasibility, effectiveness and extensibility of our approach.
Facial features, Face recognition, Data models, Mouth, Three-dimensional displays, Training, Lighting
M. X. Huang, G. Ngai, K. A. Hua, S. C. Chan and H. V. Leong, "Identifying User-Specific Facial Affects from Spontaneous Expressions with Minimal Annotation," in IEEE Transactions on Affective Computing, vol. 7, no. 4, pp. 360-373, 2016.