The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan.-March (2013 vol.4)
pp: 15-33
Andrea Kleinsmith , University of London, London
Nadia Bianchi-Berthouze , UCL Interaction Centre, London
ABSTRACT
Thanks to the decreasing cost of whole-body sensing technology and its increasing reliability, there is an increasing interest in, and understanding of, the role played by body expressions as a powerful affective communication channel. The aim of this survey is to review the literature on affective body expression perception and recognition. One issue is whether there are universal aspects to affect expression perception and recognition models or if they are affected by human factors such as culture. Next, we discuss the difference between form and movement information as studies have shown that they are governed by separate pathways in the brain. We also review psychological studies that have investigated bodily configurations to evaluate if specific features can be identified that contribute to the recognition of specific affective states. The survey then turns to automatic affect recognition systems using body expressions as at least one input modality. The survey ends by raising open questions on data collecting, labeling, modeling, and setting benchmarks for comparing automatic recognition systems.
INDEX TERMS
Face recognition, Emotion recognition, Data models, Cultural differences, cross-cultural differences, Face recognition, Emotion recognition, Data models, Cultural differences, spatiotemporal affective body features, Affective body posture, affective body movement, affective recognition systems
CITATION
Andrea Kleinsmith, Nadia Bianchi-Berthouze, "Affective Body Expression Perception and Recognition: A Survey", IEEE Transactions on Affective Computing, vol.4, no. 1, pp. 15-33, Jan.-March 2013, doi:10.1109/T-AFFC.2012.16
REFERENCES
[1] N. Fragopanagos and J.G. Taylor, "Emotion Recognition in Human-Computer Interaction," Neural Networks, vol. 18, no. 4, pp. 389-405, 2005.
[2] C.E. Izard, P.B. Ackerman, K.M. Schoff, and S.E. Fine, "Self-Organization of Discrete Emotions, Emotion Patterns, and Emotion-Cognition Relations," Emotion, Development, and Self-Organization: Dynamic Systems Approaches to Emotional Development, D.L. Marc and I. Granic, eds., pp. 15-36, Cambridge Univ. Press, 2002.
[3] P. Ekman and W. Friesen, Manual for the Facial Action Coding System. Consulting Psychology Press, 1978.
[4] R. von Laban, The Mastery of Movement. MacDonald & Evans Ltd., 1971.
[5] R.H. Rozensky and L. Feldman-Honor, "Notation Systems for Coding Nonverbal Behavior: A Review," J. Behavioral Assessment, vol. 4, no. 2, pp. 119-132, 1982.
[6] A. Mehrabian and J. Friar, "Encoding of Attitude by a Seated Communicator via Posture and Position Cues," J. Consulting and Clinical Psychology, vol. 33, pp. 330-336, 1969.
[7] M. Argyle, Bodily Communication. Methuen & Co. Ltd., 1988.
[8] P.E. Bull, Posture and Gesture. Pergamon, 1987.
[9] P. Ekman and W. Friesen, "Detecting Deception from the Body or Face," J. Personality and Social Psychology, vol. 29, no. 3, pp. 288-298, 1974.
[10] L. McClenney and R. Neiss, "Post-Hypnotic Suggestion: A Method for the Study of Nonverbal Communication," J. Nonverbal Behavior, vol. 13, pp. 37-45, 1989.
[11] H. Meeren, C. van Heijnsbergen, and B. de Gelder, "Rapid Perceptual Integration of Facial Expression and Emotional Body Language," Proc. Nat'l Academy of Sciences USA, vol. 102, no. 45, pp. 16518-16523, 2005.
[12] J. Van den Stock, R. Righart, and B. de Gelder, "Body Expressions Influence Recognition of Emotions in the Face and Voice," Emotion, vol. 7, no. 3, pp. 487-494, 2007.
[13] A.N. Antle, G. Corness, and M. Droumeva, "What the Body Knows: Exploring the Benefits of Embodied Metaphorsin Hybrid Physical Digital Environments," Interacting with Computers, vol. 21, pp. 66-75, 2009.
[14] N. Bianchi-Berthouze, "Understanding the Role of Body Movement in Player Engagement," Human-Computer Interaction, in press, http://web4.cs.ucl.ac.uk/uclic/people/n.berthouze BerthouzeHCI12.pdf
[15] F. Muller and N. Bianchi-Berthouze, "Evaluating Exertion Games," Evaluating User Experience in Games, HCI Series, Part 4, pp. 187-207, Springer 2010.
[16] P.M. Niedenthal, L.W. Barsalou, P. Winkielman, S. Krauth-Gruber, and F. Ric, "Embodiment in Attitudes, Social Perception, and Emotion," Personality and Social Psychology Rev., vol. 9, pp. 184-211, 2005.
[17] L. Barsalou, "Grounded Cognition," Ann. Rev. Psychogy, vol. 59, pp. 617-645, 2008.
[18] S. Goldin-Meadow, H. Nusbaum, S.D. Kelly, and S. Wagner, "Explaining Math: Gesturing Lightens the Load," Psychological Science, vol. 12, no. 6, pp. 516-522, 2001.
[19] J. Chandler and N. Schwarz, "How Extending Your Middle Finger Affects Your Perception of Others: Learned Movements Influence Concept Accessibility," J. Experimental Social Psychology, vol. 45, no. 1, pp. 123-128, 2009.
[20] S. Neill and C. Caswell, Body Language for Competent Teachers. Routledge, 1993.
[21] D.H. Jonassen and B.L. Grabowski, Handbook of Individual Differences, Learning, and Instruction. Erlbaum, 1993.
[22] A. Kapoor, W. Burleson, and R.W. Picard, "Automatic Prediction of Frustration," Int'l J. Human-Computer Studies, vol. 65, no. 8, pp. 724-736, 2007.
[23] A. Kvåle, A.E. Ljunggren, and T.B. Johnsen, "Examination of Movement in Patients with Long-Lasting Musculoskeletal Pain: Reliability and Validity," Physiotherapy Research Int'l, vol. 8, pp. 36-52, 2003.
[24] G.K. Haugstad, T.S. Haugstad, U.M. Kirste, S. Leganger, S. Wojniusz, I. Klemmetsen, and U.F. Malt, "Posture, Movement Patterns, and Body Awareness in Women with Chronic Pelvic Pain," J. Psychosomatic Research, vol. 61, no. 5, pp. 637-644, 2006.
[25] B. Bunkan, A.E. Ljunggren, S. Opjordsmoen, O. Moen, and S. Friis, "What Are the Dimensions of Movement?" Nord J. Psychiatry, vol. 55, pp. 33-40, 2001.
[26] J.W.S. Vlaeyen and S.J. Linton, "Fear-Avoidance and Its Consequences in Muscleskeleton Pain: A State of the Art," Pain, vol. 85, no. 3, pp. 317-332, 2000.
[27] M.M. Gross, E.A. Crane, and B.L. Fredrickson, "Methodology for Assessing Bodily Expression of Emotion," J. Nonverbal Behavior, vol. 34, pp. 223-248, 2010.
[28] H. Gunes and M. Pantic, "Automatic, Dimensional and Continuous Emotion Recognition," Int'l J. Synthetic Emotion, vol. 1, no. 1, pp. 68-99, 2010.
[29] D. McNeill, Hand and Mind—What Gestures Reveal about Thought. The Univ. of Chicago Press, 1992.
[30] B. DeCarolis, C. Pelachaud, I. Poggi, and M. Steedman, "APML, a Mark-Up Language for Believable Behavior Generation," Life-Like Characters: Tools, Affective Functions and Applications, H. Prendinger, M. Ishizuka, eds., pp. 65-85, Springer, 2004.
[31] J. Cassell, "Body Language: Lessons from the Near-Human," Genesis Redux: Essays on the History and Philosophy of Artificial Life. J. Riskin, ed., pp. 346-374, Univ. of Chicago Press, 2007.
[32] C. Pelachaud, "Studies on Gesture Expressivity for a Virtual Agent," Speech Comm., vol. 51, no. 7, pp. 630-639, 2009.
[33] R. Picard, "Toward Agents that Recognize Emotion," Proc. IMAGINA, pp. 153-165, 1998.
[34] B. de Gelder, "Why Bodies? Twelve Reasons for Including Bodily Expressions in Affective Neuroscience," Philosophical Trans. Royal Soc., vol. 364, no. 3, pp. 3475-3484, 2009.
[35] W. Zhao, R. Chellappa, and A. Rosenfeld, "Face Recognition: A Literature Survey," ACM Computing Surveys, vol. 35, pp. 399-458, 2003.
[36] M. Pantic and L.J.M. Rothkrantz, "Automatic Analysis of Facial Expressions: The State of the Art," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424-1445, Dec. 2000.
[37] B. de Gelder, "Towards the Neurobiology of Emotional Body Language," Nature Rev. Neuroscience, vol. 7, no. 3, pp. 242-249, 2006.
[38] P. Ekman and W. Friesen, "Felt, False and Miserable Smiles," J. Nonverbal Behavior, vol. 6, no. 4, pp. 238-252, 1982.
[39] R. Adolphs, "Neural Systems for Recognizing Emotion," Current Opinion in Neurobiology, vol. 12, no. 2, pp. 169-177, 2002.
[40] H.G. Wallbott, "Bodily Expression of Emotion," European J. Social Psychology, vol. 28, pp. 879-896, 1998.
[41] P. Ekman and W. Friesen, "Head and Body Cues in the Judgment of Emotion: A Reformulation," Perceptual and Motor Skills, vol. 24, pp. 711-724, 1967.
[42] P. Ekman and W. Friesen, "The Repertoire of Non-Verbal Behavioral Categories: Origins, Usage and Coding," Semiotica, vol. 1, pp. 49-98, 1969.
[43] C. Darwin, The Expression of the Emotions in Man and Animals. Murray, 1872.
[44] P. Ekman and W. Friesen, "Nonverbal Leakage and Clues to Deception," Psychiatry, vol. 32, pp. 88-105, 1969.
[45] P. Ekman, "Mistakes When Deceiving," Annals New York Academy of Sciences, vol. 364, pp. 269-278, 1981.
[46] J.E. Hocking and D.G. Leathers, "Nonverbal Indicators of Deception: A New Theoretical Perspective," Comm. Monographs, vol. 4, no. 2, pp. 119-131, 1980.
[47] R. Kraut, "Verbal and Nonverbal Cues in the Perception of Lying," J. Personality and Social Psychology, vol. 36, no. 4, pp. 380-391, 1978.
[48] J.E. Hocking, "Detecting Deceptive Communication from Verbal, Visual, and Paralinguistic Cues: An Exploratory Experiment," unpublished doctoral dissertation, Michigan State Univ., 1976.
[49] G.E. Littlepage and M.A. Pineault, "Detection of Deceptive Factual Statements from the Body and the Face," Personality and Social Psychology Bull., vol. 5, pp. 325-328, 1979.
[50] R.E. Riggio and H.S. Friedman, "Individual Differences and Cues to Deception," J. Personality and Social Psychology, vol. 45, no. 4, pp. 899-915, 1983.
[51] P. Ekman and W. Friesen, Pictures of Facial Affect. Consulting Psychologists Press, 1976.
[52] N. Hadjikhani and B. de Gelder, "Seeing Fearful Body Expressions Activates the Fusiform Cortex and Amygdala," Current Biology, vol. 13, pp. 2201-2205, 2003.
[53] B. de Gelder, J. Snyder, D. Greve, G. Gerard, and N. Hadjikhani, "Fear Fosters Flight: A Mechanism for Fear Contagion When Perceiving Emotion Expressed by a Whole Body," Proc. Nat'l Academy of Sciences USA, vol. 101, no. 47, pp. 16701-16706, 2004.
[54] H.M. Paterson, F.E. Pollick, and E. Jackson, "Movement and Faces in the Perception of Emotion from Motion," Perception, vol. 31, no. 118, pp. 232-232, 2002.
[55] F.E. Pollick, H. Paterson, and P. Mamassian, "Combining Faces and Movements to Recognize Affect," J. Vision, vol. 4, no. 8, pp. 232-232, 2004.
[56] R. Tuminello and D. Davidson, "What the Face and Body Reveal: In-Group Emotion Effects and Stereotyping of Emotion in African-American and European-American Children," J. Experimental Child Psychology, special issue on rmotion assessment in children and adolescents, vol. 110, pp. 258-274, 2011.
[57] M.L. Willis, R. Palermo, D. Burke, "Judging Approachability on the Face of It: The Influence of Face and Body Expressions on the Perception of Approachability," Emotion, vol. 11, no. 3, pp. 514-523, 2011.
[58] H.M. Paterson, F.E. Pollick, and A.J. Sanford, "The Role of Velocity in Affect Discrimination," Proc. 23rd Ann. Conf. Cognitive Science Soc., pp. 756-761, 2001.
[59] A. Kleinsmith and N. Bianchi-Berthouze, "Grounding Affective Dimensions into Posture Features," Proc. First Int'l Conf. Affective Computing and Intelligent Interaction, pp. 263-270, 2005.
[60] A. Kleinsmith, N. Bianchi-Berthouze, and A. Steed, "Automatic Recognition of Non-Acted Affective Postures," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 41, no. 4, pp. 1027-1038, Aug. 2011.
[61] M. Karg, K. Kuhnlenz, and M. Buss, "Recognition of Affect Based on Gait Patterns," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 40, no. 4, pp. 1050-1061, Aug. 2010.
[62] J.K. Burgoon, M.L. Jensen, T.O. Meservy, J. Kruse, and J.F. Nunamaker, "Augmenting Human Identification of Emotional States in Video," Proc. Int'l Conf. Intelligent Data Analysis, 2005.
[63] D. Matsumoto, "Culture and Nonverbal Behavior," Handbook of Nonverbal Comm., V. Manusov and M. Patterson, eds., Sage, 2005.
[64] S. Kopp, L. Gesellensetter, N. Kramer, and I. Wachsmuth, "A Conversational Agent as Museum Guide—Design and Evaluation of a Real-World Application," Proc. Fifth Int'l Workshop Intelligent Virtual Agents, J. Tao, T. Tan, and R. Picard, eds., pp. 329-343, 2005.
[65] W. Swartout, D. Traum, R. Artstein, D. Noren, P. Debevec, K. Bronnenkant, J. Williams, A. Leuski, S. Narayanan, D. Piepol, C. Lane, J. Morie, P. Aggar wal, M. Liewer, J-Y. Chiang, J. Gerten, S. Chu, and K. White, "Ada and Grace: Toward Realistic and Engaging Virtual Museum Guides," Proc. Int'l Conf. Intelligent Virtual Agents, pp. 286-300, 2010.
[66] P. Dunn and A. Marinetti, Cultural Adaptation: Necessity for eLearning, http://www.linezine.com/7.2/articlespdamca.htm , Retrieved Jan. 2006.
[67] D. Keltner, P. Ekman, G.C. Gonzaga, and J. Beer, "Facial Expression of Emotion," Handbook of Affective Sciences, R. Davidson, K. Scherer, and H. Goldsmith, eds., Oxford Univ. Press, 2003.
[68] H.A. Elfenbein, M.K. Mandal, N. Ambady, S. Harizuka, and S. Kumar, "Cross-Cultural Patterns in Emotion Recognition: Highlighting Design and Analytical Techniques," Emotion, vol. 2, no. 1, pp. 75-84, 2002.
[69] P. Ekman, "Strong Evidence for Universals in Facial Expressions: A Reply to Russell's Mistaken Critique," Psychological Bull., vol. 115, pp. 268-287, 1994.
[70] L. Yin, X. Wei, Y. Sun, J. Wang, and M.J. Rosato, The Binghamton University 3D Facial Expression Data Base, http:/www.cs. binghamton.edu/, 2012.
[71] J.A. Russell, "Is There Universal Recognition of Emotion from Facial Expressions? A Review of the Cross-Cultural Studies," Psychological Bull., vol. 115, pp. 102-141, 1994.
[72] B. Mesquita, "Emotions as Dynamic Cultural Phenomena," Handbook of Affective Sciences, R. Davidson, K. Scherer, and H. Goldsmith, eds., Oxford Univ. Press, 2003.
[73] H. Elfenbein and N. Ambady, "On the Universality and Cultural Specificity of Emotion Recognition: A Metaanalysis," Psychological Bull., vol. 128, pp. 205-235, 2002.
[74] L. Ducci, L. Arcuri, T. Georgis, and T. Sineshaw, "Emotion Recognition in Ethiopia: The Effects of Familiarity with Western Culture on Accuracy of Recognition," J. Cross-Cultural Psychology, vol. 13, pp. 340-351, 1982.
[75] U. Hess, S. Senecal, and G. Kirouac, "Recognizing Emotional Facial Expressions: Does Perceived Sociolinguistic Group Make a Difference?" Int'l J. Psychology, vol. 31, p. 93, 1996.
[76] T. Kudoh and D. Matsumoto, "Cross-Cultural Examination of the Semantic Dimensions of Body Postures," J. Personality Social Psychology, vol. 48, no. 6, pp. 1440-1446, 1985.
[77] D. Matsumoto and T. Kudoh, "Cultural Similarities and Differences in the Semantic Dimensions of Body Postures," J. Nonverbal Behavior, vol. 11, no. 3, pp. 166-179, 1987.
[78] M.H. Bond, H. Nakazato, and D. Shiraishi, "Universality and Distinctiveness in Dimensions of Japanese Person Perception," J. Cross-Cultural Psychology, vol. 6, pp. 346-357, 1975.
[79] D. Matsumoto and H. Kishimoto, "Developmental Characteristics in Judgments of Emotion from Nonverbal Vocal Cues," Int'l J. Intercultural Relations, vol. 7, pp. 415-424, 1983.
[80] A. Kleinsmith, P.R. de Silva, and N. Bianchi-Berthouze, "Cross-Cultural Differences in Recognizing Affect from Body Posture," Interacting with Computers, vol. 18, pp. 1371-1389, 2006.
[81] M. Brandt and J. Boucher, "Concepts of Depression in Emotion Lexicons of Eight Cultures," Int'l J. Intercultural Relations, vol. 10, pp. 321-346, 1986.
[82] D. Matsumoto, T. Consolacion, H. Yamada, R. Suzuki, B. Franklin, S. Paul, R. Ray, and H. Uchida, "American-Japanese Cultural Differences in Judgments of Emotional Expressions of Different Intensities," Cognition and Emotion, vol. 16, pp. 721-747, 2002.
[83] M.A. Giese and T. Poggio, "Neural Mechanisms for the Recognition of Biological Movements," Neuroscience, vol. 4, pp. 179-191, 2003.
[84] L.M. Vania, M. Lemay, D.C. Bienfang, A.Y. Choi, and K. Nakayama, "Intact Biological Motion and Structure from Motion Perception in a Patient with Impaired Motion Mechanisms: A Case Study," Visual Neuroscience, vol. 5, pp. 353-369, 1990.
[85] J. Lange and M. Lappe, "The Role of Spatial and Temporal Information in Biological Motion Perception," Advances in Cognitive Psychology, vol. 3, no. 4, pp. 419-428, 2007.
[86] M. Hirai and K. Hiraki, "The Relative Importance of Spatial versus Temporal Structure in the Perception of Biological Motion: An Event-Related Potential Study," Cognition, vol. 99, pp. B15-B29, 2006.
[87] M.V. Peelen, A.J. Wiggett, and P.E. Downing, "Patterns of fMRI Activity Dissociate Overlapping Functional Brain Areas that Respond to Biological Motion," Neuron, vol. 49, pp. 815-822, 2006.
[88] P. McLeod, W. Dittrich, J. Driver, D. Perret, and J. Zihl, "Preserved and Impaired Detection of Structure from Motion by a 'Motion-Blind' Patient," Visual Cognition, vol. 3, pp. 363-391, 1996.
[89] A.P. Atkinson, W.H. Dittrich, A.J. Gemmell, and A.W. Young, "Evidence for Distinct Contributions of Form and Motion Information to the Recognition of Emotions from Body Gestures," Cognition, vol. 104, pp. 59-72, 2007.
[90] L. Omlor and M.A. Giese, "Extraction of Spatio-Temporal Primitives of Emotional Body Expressions," Neurocomputing, vol. 70, nos. 10-12, pp. 1938-1942, 2007.
[91] C. Roether, L. Omlor, A. Christensen, and M.A. Giese, "Critical Features for the Perception of Emotion from Gait," J. Vision, vol. 8, nos. 6-15, pp. 1-32, 2009.
[92] J. Aronoff, B.A. Woike, and L.M. Hyman, "Which Are the Stimuli in Facial Displays of Anger and Happiness? Configurational Bases of Emotion Recognition," J. Personality and Social Psychology, vol. 62, pp. 1050-1066, 1992.
[93] J.A. Harrigan and R. Rosenthal, "Physicians Head and Body Positions as Determinants of Perceived Rapport," J. Applied Social Psychology, vol. 13, no. 6, pp. 496-509, 1983.
[94] A. Mehrabian, "Inference of Attitude from the Posture, Orientation, and Distance of a Communicator," J. Consulting and Clinical Psychology, vol. 32, pp. 296-308, 1968.
[95] W.T. James, "A Study of the Expression of Bodily Posture," J. General Psychology, vol. 7, pp. 405-437, 1932.
[96] S. Dahl and A. Friberg, "Visual Perception of Expressiveness in Musicians' Body Movements," Music Perception, vol. 24, no. 5, pp. 433-454, 2007.
[97] G. Castellano, M. Mortillaro, A. Camurri, G. Volpe, and K. Scherer, "Automated Analysis of Body Movement in Emotionally Expressive Piano Performances," Music Perception, vol. 26, no. 2, pp. 103-120, 2008.
[98] D. Glowinski, N. Dael, A. Camurri, G. Volpe, M. Mortillaro, and K. Scherer, "Towards a Minimal Representation of Affective Gestures," IEEE Trans. Affective Computing, vol. 2, no. 2, pp. 106-118, Apr.-June 2011.
[99] T. Banziger and K.R. Scherer, "Introducing the Geneva Multimodal Emotion Portrayal Corpus," Blueprint for Affective Computing: A Sourcebook, pp. 271-294, Oxford Univ. Press, 2010.
[100] M. Coulson, "Attributing Emotion to Static Body Postures: Recognition Accuracy, Confusions, and Viewpoint Dependence," J. Nonverbal Behavior, vol. 28, pp. 117-139, 2004.
[101] M. de Meijer, "The Contribution of General Features of Body Movement to the Attribution of Emotions," J. Nonverbal Behavior, vol. 13, pp. 247-268, 1989.
[102] R. De Silva and N. Bianchi-Berthouze, "Modeling Human Affective Postures: An Information Theoretic Characterization of Posture Features," J. Computer Animation and Virtual Worlds, vol. 15, nos. 3/4, pp. 269-276, 2004.
[103] A. Kleinsmith and N. Bianchi-Berthouze, "Recognizing Affective Dimensions from Body Posture," Proc. Int'l Conf. Affective Computing and Intelligent Interaction, pp. 48-58, 2007.
[104] R.A. Calvo and S. D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications," IEEE Trans. Affective Computing, vol. 1, no. 1, pp. 18-37, Jan.-June 2010.
[105] H. Gunes, M. Piccardi, and M. Pantic, "From the Lab to the Real World: Affect Recognition Using Multiple Cues and Modalities," Affective Computing: Focus on Emotion Expression, Synthesis, and Recognition, J. Or, ed., pp. 185-218, I-Tech Educational Publishing, 2008.
[106] M. Pantic and L.J.M. Rothkrantz, "Automatic Analysis of Facial Expressions: The State of the Art," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424-1445, Dec. 2000.
[107] M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 568-573, 2005.
[108] M. Rosenblum, Y. Yacoob, and L.S. Davis, "Human Emotion Recognition from Motion Using a Radial Basis Function Network Architecture," Proc. Workshop Motion of Non-Rigid and Articulated Objects, 1994.
[109] P.Y. Oudeyer, "The Production and Recognition of Emotions in Speech: Features and Algorithms," Int'l J. Human-Computer Studies, vol. 59, pp. 157-183, 2003.
[110] V. Kostov and S. Fukuda, "Emotion in User Interface, Voice Interaction System," Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, pp. 798-803, 2007.
[111] C. Lee, S. Narayanan, and R. Pieraccini, "Classifying Emotions in Human-Machine Spoken Dialogs," Proc. IEEE Int'l Conf. Multimedia and Expo, 2002.
[112] V. Petrushin, "Emotion in Speech: Recognition and Application to Call Centers," Proc. Artificial Neural Networks in Eng. Conf., pp. 7-10, 1999.
[113] A. Camurri, B. Mazzarino, M. Ricchetti, R. Timmers, and G. Volpe, "Multimodal Analysis of Expressive Gesture in Music and Dance Performances," Gesture-Based Comm. in HCI, vol. 2915, pp. 20-39, 2004.
[114] A. Kapur, A. Kapur, N. Virji-Babul, G. Tzanetakis, and P.F. Driessen, "Gesture-Based Affective Computing on Motion Capture Data," Proc. First Int'l Conf. Affective Computing and Intelligent Interaction, pp. 1-7, 2005.
[115] F.E. Pollick, V. Lestou, J. Ryu, and S-B. Cho, "Estimating the Efficiency of Recognizing Gender and Affect from Biological Motion," Vision Research, vol. 42, pp. 2345-2355, 2002.
[116] D. Bernhardt and P. Robinson, "Detecting Affect from Non-Stylised Body Motions," Proc. Second Int'l Conf. Affective Computing and Intelligent Interaction, pp. 59-70, 2007.
[117] N. Bianchi-Berthouze and A. Kleinsmith, "A Categorical Approach to Affective Gesture Recognition," Connection Science, vol. 15, pp. 259-269, 2003.
[118] N. Savva, A. Scarinzi, and N. Bianchi-Berthouze, "Continuous Recognition of Player's Affective Body Expression as Dynamic Quality of Aesthetic Experience," IEEE Trans. Computational Intelligence and AI in Games, vol. 4, no. 3, pp. 199-212, 2011.
[119] L. Gong, T. Wang, C. Wang, F. Liu, F. Zhang, and X. Yu, "Recognizing Affect from Non-Stylized Body Motion Using Shape of Gaussian Descriptors," Proc. ACM Symp. Applied Computing, 2010.
[120] D. Bernhardt and P. Robinson, "Detecting Emotions from Connected Action Sequences, Visual Informatics: Bridging Research and Practice," Proc. First Int'l Visual Informatics Conf. Visual Informatics: Bridging Research and Practice, pp. 1-11, 2009.
[121] A. Kapoor, R.W. Picard, and Y. Ivanov, "Probabilistic Combination of Multiple Modalities to Detect Interest," Proc. 17th Int'l Conf. Pattern Recognition, vol. 3, pp. 969-972, 2004.
[122] H. Gunes and M. Piccardi, "Bi-Modal Emotion Recognition from Expressive Face and Body Gestures," J. Network and Computer Applications, vol. 30, pp. 1334-1345, 2007.
[123] H. Gunes and M. Piccardi, "Automatic Temporal Segment Detection and Affect Recognition from Face and Body Display," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 39, no. 1, pp. 64-84, Feb. 2009.
[124] C.F. Shan, S.G. Gong, and P.W. McOwan, "Beyond Facial Expressions: Learning Human Emotion from Body Gestures," Proc. British Machine Vision Conf., 2007.
[125] H. Park, J. Park, U. Kim, and W. Woo, "Emotion Recognition from Dance Image Sequences using Contour Approximation," Proc. Int'l Workshop Structural, Syntactic, and Statistical Pattern Recognition, pp. 547-555, 2004.
[126] A. Camurri, R. Trocca, and G. Volpe, "Interactive Systems Design: A KANSEI-Based Approach," Proc. Conf. New Interfaces for Musical Expression, pp. 1-8, 2002.
[127] S. Kamisato, S. Odo, Y. Ishikawa, and K. Hoshino, "Extraction of Motion Characteristics Corresponding to Sensitivity Information Using Dance Movement," J. Advanced Computational Intelligence and Intelligent Informatics, vol. 8, no. 2, pp. 167-178, 2004.
[128] A. Camurri, I. Lagerlof, and G. Volpe, "Recognizing Emotion from Dance Movement: Comparison of Spectator Recognition and Automated Techniques," Int'l J. Human-Computer Studies, vol. 59, nos. 1/2, pp. 213-225, 2003.
[129] R. von Laban, Modern Educational Dance. MacDonald & Evans, Ltd., 1963.
[130] Y. Ma, H.M. Paterson, and F.E. Pollick, "A Motion Capture Library for the Study of Identity, Gender, and Emotion Perception from Biological Motion," Behavior Research Methods, vol. 38, no. 1, pp. 134-141, 2006.
[131] F.E. Pollick, H.M. Paterson, A. Bruderlin, and A.J. Sanford, "Perceiving Affect from Arm Movement," Cognition, vol. 82, pp. 51-61, 2001.
[132] J. Sanghvi, G. Castellano, I. Leite, A. Pereira, P.W. McOwan, and A. Paiva, "Automatic Analysis of Affective Postures and Body Motion to Detect Engagement with a Game Companion," Proc. Int'l Conf. Human-Robot Interaction, 2011.
[133] A. Kapoor, S. Mota, and R.W. Picard, "Towards a Learning Companion that Recognizes Affect," Technical Report 543, MIT Media Lab, 2001.
[134] R. el Kaliouby and P. Robinson, "Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures," Proc. IEEE Int'l Workshop Real Time Computer Vision for Human Computer Interaction at CVPR, 2004.
[135] G. Varni, G. Volpe, and A. Camurri, "A System for Real-Time Multimodal Analysis of Nonverbal Affective Social Interaction in User-Centric Media," IEEE Trans. Multimedia, vol. 12, no. 6, pp. 576-590, Oct. 2010.
[136] W.H. Dittrich, "Action Categories and the Perception of Biological Motion," Perception, vol. 22, no. 1, pp. 15-22, 1993.
[137] A.P. Atkinson, "Impaired Recognition of Emotions from Body Movements is Associated with Elevated Motion Coherence Thresholds in Autism Spectrum Disorders," Neuropsychologia, vol. 47, no. 13, pp. 3023-3029, 2009.
[138] H.L. Gallagher and C.D. Frith, "Dissociable Neural Pathways for the Perception and Recognition of Expressive and Instrumental Gestures," Neuropsychologia, vol. 42, no. 13, pp. 1725-1736, 2004.
[139] A.J. Calder, A.M. Burton, P. Miller, A.W. Young, and S. Akamatsu, "A Principal Component Analysis of Facial Expressions," Vision Research, vol. 41, no. 9, pp. 1179-1208, 2001.
[140] U. Martens, H. Leuthold, and S.R. Schweinberger, "On the Temporal Organization of Facial Identity and Expression Analysis: Inferences from Event-Related Brain Potentials," Cognitive, Affective, and Behavioral Neuroscience, vol. 10, no. 4, pp. 505-522, 2010.
[141] P. Ekman and W.V. Friesen, Pictures of Facial Affect. Consulting Psychologists Press, http://www.paulekman.com researchproducts.phpt , Oct. 2008.
[142] J. Fürnkranz and E. HÄullermeier, "Preference Learning," Kunstliche Intelligenz, vol. 19, no. 1, pp. 60-61, 2005.
[143] J. Doyle, "Prospects for Preferences," Computational Intelligence, vol. 20, no. 2, pp. 111-136, 2004.
[144] G.N. Yannakakis, "Preference Learning for Affective Modelling," Proc. Third Int'l Conf. Affective Computing and Intelligent Interaction, pp. 126-131, 2009.
[145] H. Meng, A. Kleinsmith, and N. Bianchi-Berthouze, "Multi-Score Learning for Affect Recognition: the Case of Body Postures," Proc. Fourth Int'l Conf. Affective Computing and Intelligent Interaction, 2011.
[146] A. Mehrabian and N. Epstein, "A Measure of Emotional Empathy," J. Personality, vol. 40, pp. 525-543, 1972.
[147] V.S. Sheng, F. Provost, and P.G. Ipeirotis, "Get Another Label? Improving Data Quality and Data Mining Using Multiple, Noisy Labellers," Proc. 14th ACM SIGKDD Int'l Conf. Knowledge Disc and Data Mining, pp. 614-622, 2008.
[148] V. Raykar, S. Yu, L. Zhao, G. Valadez, C. Florin, L. Bogoni, and L. Moy, "Learning from Crowds," J. Machine Learning Research, vol. 11, no. 7, pp. 1297-1322, 2010.
[149] J. Howe, Crowd Sourcing: Why the Power of the Crowd Is Driving the Future of Business. Crown Publishing Group 2009.
[150] A. Tarasov, C. Cullen, and S.J. Delany, "Using Crowdsourcing for Labelling Emotional Speech Assets," Proc. W3C Workshop Emotion Markup Language, 2010.
[151] P. Juslin and P. Laukka, "Communication of Emotions in Vocal Expression and Music Performance," Psychological Bull., vol. 129, no. 5, pp. 770-814, 2002.
[152] M. Kienast and W.F. Sendlmeier, "Acoustical Analysis of Spectral and Temporal Changes in Emotional Speech," Proc. ISCA Workshop Speech and Emotion, pp. 92-97, 2000.
[153] L. Leinonen and T. Hiltunen, "Expression of Emotional-Motivational Connotations with a One-Word Utterance," J. Acoustical Soc. of Am., vol. 102, no. 3, pp. 1853-1863, 1997.
[154] S. Yacoub, S. Simske, X. Lin, and J. Burns, "Recognition of Emotions in Interactive Voice Response Systems," Eurospeech: Proc. Eighth European Conf. Speech Comm. and Technology, 2003.
[155] H.G. Wallbott and K.R. Scherer, "Cues and Channels in Emotion Recognition," J. Personality and Social Psychological, vol. 51, no. 4, pp. 690-699, 1986.
[156] N. Sebe, Y. Sun, E. Bakker, M.S. Lew, I. Cohen, and T.S. Huang, "Towards Authentic Emotion Recognition," Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, pp. 623-628, 2004.
[157] T. Bänziger and K.R. Scherer, "Using Actor Portrayals to Systematically Study Multimodal Emotion Expression: The GEMEP Corpus," Proc. Int'l Conf. Affective Computing and Intelligence Interaction, pp. 476-487, 2007.
[158] G. Castellano, I. Leite, A. Pereira, C. Martinho, A. Paiva, and P. McOwan, "Affect Recognition for Interactive Companions: Challenges and Design in Real-World Scenarios," J. Multimodal User Interfaces, vol. 3, nos. 1/2, pp. 89-98, 2010.
[159] A. Ashraf, S. Lucey, J. Cohn, T. Chen, K. Prkachin, and P. Solomon, "The Painful Face: Pain Expression Recognition Using Active Appearance Models," Image and Vision Computing, vol. 27, pp. 1788-1796, 2009.
[160] S. Afzal and P. Robinson, "Natural Affect Data—Collection and Annotation in a Learning Context," Proc. Third Int'l Conf. Affective Computing and Intelligent Interaction, pp. 22-28, 2009.
[161] N. Dael, M. Mortillaro, and K.R. Scherer, "Introducing the Body Action and Posture Coding System (BAP): Development and Reliability," manuscript dubmitted for publication.
[162] H. Gunes and M. Piccardi, "A Bimodal Face and Body Gesture Database for Automatic Analysis of Human Nonverbal Affective Behavior," Proc. 18th Int'l Conf. Pattern Recognition, pp. 1148-1153, 2006.
[163] M. Karg, R. Jenke, W. Seiberl, K. Kuhnlenz, A. Schwirtz, and M. Buss, "A Comparison of PCA, KPCA, and LDA for Feature Extraction to Recognize Affect in Gait Patterns," Proc. Third In'l Conf. Affective Computing and Intelligent Interaction, pp. 195-200, 2009.
[164] P.J. Watson, C.K. Booker, C.J. Main, and A.C. Chen, "Surface Electromyography in the Identification of Chronic Low Back Pain Patients: The Development of the Flexion Relaxation Ratio," Clinical Biomechanics, vol. 12, no. 3, pp. 165-71, 1997.
[165] M. Pluess, A. Conrad, and F.H. Wilhelm, "Muscle Tension in Generalized Anxiety Disorder: A Critical Review of the Literature," J. Anxiety Disorders, vol. 23, no. 1, pp. 1-11, 2009.
[166] M.E. Geisser, A.J. Haig, A.S. Wallbom, and E.A. Wiggert, "Pain-Related Fear, Lumbar Flexion, and Dynamic EMG among Persons with Chronic Musculoskeletal Low Back Pain," Clinical J. Pain, vol. 20, no. 2, pp. 61-69, 2004.
[167] M. Kamachi, M. Lyons, and J. Gyoba, The Japanese Female Facial Expression Database, http://www.kasrl.orgjaffe.html, retrieved Oct. 2008.
[168] M. Pantic, M. Valstar, R. Rademaker, and L. Maat, "Web-Based Database for Facial Expression Analysis," Proc. IEEE Int'l Conf. Multimedia and Expo, 2005.
[169] M. Pantic, M. Valstar, R. Rademaker, and L. Maat, The MMI Face Database, http:/www.mmifacedb.com/, retrieved, Oct. 2008.
[170] S. Baron-Cohen, O. Golan, S. Wheelright, and J. Hill, The Mindreading DVD, http://www.jkp.commindreading/, retrieved Oct. 2008.
[171] The Psychological Image Collection at Stirling. http:/pics.psych. stir.ac.uk/, retrieved Oct. 2008.
[172] Center for the Study of Emotion and Attention. The International Affective Picture System. http://csea.phhp.udeg;.edu/mediaiaps message.html , Retrieved, Oct. 2008.
[173] T. Kanade, J.F. Cohn, and Y. Tian, "Comprehensive Database for Facial Expression Analysis," Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 46-53, 2000.
[174] J.K. Burgoon, M.L. Jensen, T.O. Meservy, J. Kruse, and J.F. NunamakerJr., "Augmenting Human Identification of Emotional States in Video," Proc. Int'l Conf. Intelligent Data Analysis, 2005.
[175] P. Ekman, "Lying and Deception," Memory for Everyday and Emotional Events, N.L. Stein, P.A. Ornstein, B. Tversky, and C. Brainerd, Lawrence Erlbaum Assoc., 1997.
[176] B.M. DePaulo, J.J. Lindsay, B.E. Malone, L. Muhlenbruck, K. Charlton, and H. Cooper, "Cues to Deception," Psychological Bull., vol. 129, pp.74-112, 2003.
[177] H. Gunes and M. Piccardi, "Observer Annotation of Affective Display and Evaluation of Expressivity: Face versus Face-and-Body," Proc. HCSNet Workshop Use of Vision in HCI, pp. 35-42, 2006.
[178] C. Clavel, J. Plessier, J-C. Martin, L. Ach, and B. Morel, "Combining Facial and Postural Expressions of Emotions in a Virtual Character," Proc. Ninth Int'l Conf. Intelligent Virtual Agents, pp. 387-300, 2009.
[179] D.A. Leopold, A.J. O'Toole, T. Vetter, and V. Blanz, "Prototype-Referenced Shape Encoding Revealed by High-Level Aftereffects," Nature Neuroscience, vol. 4, pp. 89-94, 2001.
[180] J.A. Russell, "Core Affect and the Psychological Construction of Emotion," Psychological Rev., vol. 110, pp. 145-172, 2003.
[181] M. Pasch, N. Bianchi-Berthouze, B. van Dijk, and A. Nijholt, "Movement-Based Sports Video Games: Investigating Motivation and Gaming Experience," Entertainment Computing, vol. 9, no. 2, pp. 169-180, 2009.
[182] J. Nijhar, N. Bianchi-Berthouze, and G. Boguslawski, "Does Movement Recognition Precision Affect the Player Experience in Exertion Games?" Proc. Int'l Conf. Intelligent Technologies for Interactive Entertainment, pp. 73-82, 2012.
[183] M. Gendron, K.A. Lindquist, L. Barsalou, and L.F. Barrett, "Emotion Words Shape Emotion Percepts," Emotion, vol. 12, pp. 314-325, 2012.
[184] K.A. Lindquist, L.F. Barrett, E. Bliss-Moreau, and J.A. Russell, "Language and the Perception of Emotion," Emotion, vol. 6, pp. 125-138, 2006.
[185] L.W. Barsalou, "Perceptual Symbol Systems," Behavioral and Brain Sciences, vol. 22, pp. 577-660, 1999.
[186] P.M. Niedenthal, L.W. Barsalou, P. Winkielman, S. Krauth-Gruber, and F. Ric, "Embodiment in Attitudes, Social Perception, and Emotion," Personality and Social Psychological Rev., vol. 9, no. 3, pp. 184-211, 2005.
[187] J. Chandler and N. Schwarz, "How Extending Your Middle Finger Affects Your Perception of Others: Learned Movements Influence Concept Accessibility," J. Experimental Social Psychology, vol. 45, no. 1, pp. 123-128, 2009.
23 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool