Issue No.02 - July-December (2010 vol.1)
pp: 132-144
Milan Gnjatović , Otto-von-Guericke-University Magdeburg, Magdeburg and University of Novi Sad, Novi Sad
Dietmar Rösner , Otto-von-Guericke-University Magdeburg, Magdeburg
Emotional corpora provide an important empirical foundation for investigation when researchers aim at implementing emotion-aware spoken dialog systems. One of the fundamental research questions is how to acquire an appropriate, realistic emotion corpus. The primary aim of this paper is to address the methodological desiderata in producing emotion corpora in human-machine interaction (HMI). It proposes a substantial refinement of the Wizard-of-Oz (WOZ) technique in order that a scenario designed to elicit affected speech in HMI could result in realistic and useful data. In addition, the paper reports about the NIMITEK corpus of affected behavior in HMI produced during a refined WOZ simulation. The evaluation of the corpus with respect to the perception of its emotional content demonstrated that the corpus contains recordings of emotions that were overtly signaled. The range of emotional reactions is indicative of the kind of emotional reactions than can be expected to occur in the interaction with the sort of spoken dialog systems considered in this study. Since the subjects were not restricted by given predetermined linguistic constraints on the language to use, their utterances are indicative of the way in which nontrained, nontechnical users probably like to converse with conversational agents as well.
Affective computing, emotional corpora, methods of data collection, methods for emotion elicitation.
Milan Gnjatović, Dietmar Rösner, "Inducing Genuine Emotions in Simulated Speech-Based Human-Machine Interaction: The NIMITEK Corpus", IEEE Transactions on Affective Computing, vol.1, no. 2, pp. 132-144, July-December 2010, doi:10.1109/T-AFFC.2010.14
[1] I. Rahwan and P. McBurney, “Argumentation Technology,” IEEE Intelligent Systems, vol. 22, no. 6, pp. 21-23, Nov./Dec. 2007.
[2] E. Douglas-Cowie and WP5 Members, “Preliminary Plans for Exemplars: Databases,” technical report, The HUMAINE Assoc., 2004.
[3] N. Fraser and G. Gilbert, “Simulating Speech Systems,” Computer Speech and Language, vol. 5, pp. 81-99, 1991.
[4] A. Batliner, K. Fischer, R. Huber, J. Spilker, and E. Nöth, “Desperately Seeking Emotions: Actors, Wizards, and Human Beings,” Proc. ISCA Workshop Speech and Emotion: A Conceptual Framework for Research, pp. 195-200, 2000.
[5] A. Batliner, C. Hacker, S. Steidl, E. Nöth, S. D'Arcy, M. Russell, and M. Wong, “‘You Stupid Tin Box’—Children Interacting with the AIBO Robot: A Cross-Linguistic Emotional Speech Corpus,” Proc. Fourth Int'l Conf. Language Resources and Evaluation, pp. 171-174, 2004.
[6] A. Batliner, S. Steidl, C. Hacker, and E. Nöth, “Private Emotions versus Social Interaction: A Data-Driven Approach towards Analysing Emotion in Speech,” User Modelling and User-Adapted Interaction—The J. Personalization Research, vol. 18, nos. 1/2, pp. 175-206, 2008.
[7] E. Douglas-Cowie, R. Cowie, C. Cox, N. Amir, and D. Heylen, “The Sensitive Artificial Listener: An Induction Technique for Generating Emotionally Coloured Conversation,” Proc. Second Int'l Workshop Corpora for Research on Emotion and Affect, pp. 1-4, 2008.
[8] K. Forbes-Riley, D. Litman, S. Silliman, and A. Purandare, “Uncertainty Corpus: Resource to Study User Affect in Complex Spoken Dialogue Systems,” Proc. Sixth Int'l Conf. Language Resources and Evaluation, 2008.
[9] V. Aharonson and N. Amir, “Emotion Elicitation in a Computerized Gambling Game,” Proc. Third Int'l Conf. Speech Prosody, pp. 179-183, 2006.
[10] A.R. Damasio, Descartes' Error: Emotion, Reason and the Human Brain. Gosset/Putnam Press, 1994.
[11] L. Devillers and J.-C. Martin, “Coding Emotional Events in Audiovisual Corpora,” Proc. Sixth Int'l Conf. Language Resources and Evaluation, 2008.
[12] M. Fék, N. Audibert, J. Szabó, A. Rilliard, G. Németh, and V. Aubergé, “Multimodal Spontaneous Expressive Speech Corpus for Hungarian,” Proc. Sixth Int'l Conf. Language Resources and Evaluation, 2008.
[13] A. Wendemuth, J. Braun, B. Michaelis, F. Ohl, D. Rösner, H. Scheich, and R. Warnemünde, “Neurobiologically Inspired, Multimodal Intention Recognition for Technical Communication Systems (NIMITEK),” Proc. Fourth IEEE Tutorial and Research Workshop Perception and Interactive Technologies for Speech-Based Systems, pp. 141-144, 2008.
[14] M. Gnjatović and D. Rösner, “On the Role of the NIMITEK Corpus in Developing an Emotion Adaptive Spoken Dialogue System,” Proc. Sixth Int'l Conf. Language Resources and Evaluation, 2008.
[15] M. Gnjatović and D. Rösner, “Adaptive Dialogue Management in the NIMITEK Prototype System,” Proc. Fourth IEEE Tutorial and Research Workshop Perception and Interactive Technologies for Speech-Based Systems, pp. 14-25, 2008.
[16] B. Wendt and H. Scheich, “The ‘Magdeburger Prosodie-Korpus’,” Proc. Int'l Conf. Speech Prosody, pp. 699-701, 2002.
[17] W. Burleson, “Affective Learning Companions: Strategies for Empathetic Agents with Real-Time Multimodal Affective Sensing to Foster Meta-Cognitive and Meta-Affective Approaches to Learning, Motivation, and Perseverance,” PhD dissertation, Massachusetts Inst. of Technology (MIT), 2006.
[18] M. Gnjatović, Adaptive Dialogue Management in Human-Machine Interaction. Verlag Dr. Hut, 2009.
[19] M. Gnjatović, M. Kunze, X. Zhang, J. Frommer, and D. Rösner, “Linguistic Expression of Emotion in Human-Machine Interaction: The NIMITEK Corpus as a Research Tool,” Proc. Fourth Int'l Workshop Human-Computer Conversation, 2008.