The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - Fourth Quarter (2012 vol.3)
pp: 482-495
Chi-Keng Wu , National Cheng Kung University, Tainan
Pau-Choo (Julia) Chung , National Cheng Kung University , Tainan
Chi-Jen Wang , National Cheng Kung University, Tainan
ABSTRACT
Human respiration has been reported as having great relevance to the emotional state. However, the respiration signal obtained using traditional biosensors not only reflects changing emotions and emotion intensities, but also contains motion artifacts. The resulting information ambiguity limits the practical application of the respiration signal as a means of reliable affective state appraisal. Thus, the present study proposes a method for extracting representative Emotion Elicited Segments (EESs) from the respiration signal such that the affective state of the individual can be more reliably determined. The EES extraction process involves the combination of the following procedures, namely, 1) Mutual Information-Based Emotion Relevance Feature Ranking based on the Dynamic Time Warping Distance (MIDTW), and 2) Constraint-based Elicited Segment Density (CESD) analysis. Due to the innate regularity of the respiration signal, the respiration pattern under a particular emotional state would be relatively quasihomogeneous over time. Accordingly, a parameter-free Respiration quasi-Homogeneity Segmentation (RHS) algorithm is proposed for partitioning the respiration signal into quasi-homogenous segments from which the EESs can then be extracted. The experimental results obtained for five prototypical emotions (i.e., “love,” “sadness,” “joy,” “anger,” and “fear”) show that the proposed segmentation/extraction methodology enables the EESs to be reliably identified and yields an average classification rate of 88 percent.
INDEX TERMS
Physiology, Feature extraction, Human factors, Motion segmentation, Algorithm design and analysis, Ethics, Partitioning algorithms, Emotion recognition, constraint-based clustering analysis, Segmentation algorithm, motion artifacts, emotion classification, dynamic time-warping, mutual information
CITATION
Chi-Keng Wu, Pau-Choo (Julia) Chung, Chi-Jen Wang, "Representative Segment-Based Emotion Analysis and Classification with Automatic Respiration Signal Segmentation", IEEE Transactions on Affective Computing, vol.3, no. 4, pp. 482-495, Fourth Quarter 2012, doi:10.1109/T-AFFC.2012.14
REFERENCES
[1] R.W. Picard, Affective Computing. MIT Press, 2000.
[2] D. Kulic and E.A. Croft, "Affective State Estimation for Human-Robot Interaction," IEEE Trans. Robotics, vol. 23, no. 5, pp. 991-1000, Oct. 2007.
[3] A. Kapoor, W. Burleson, and R.W. Picard, "Automatic Prediction of Frustration," Int'l J. Human-Computer Studies, vol. 65, no. 8, pp. 724-736, Aug. 2007.
[4] J. Scheirer, R. Fernandez, J. Klein, and R.W. Picard, "Frustrating the User on Purpose: A Step toward Building an Affective Computer," Interacting with Computers, vol. 14, no. 2, pp. 93-118, 2002.
[5] G.E. Sakr, I.H. Elhajj, and H.A.-S. Huijer, "Support Vector Machines to Define and Detect Agitation Transition," IEEE Trans. Affective Computing, vol. 1, no. 2, pp. 98-108, July-Dec. 2010.
[6] L.S. Greenberg, "The Clinical Application of Emotion in Psychotherapy," Handbook of Emotions, M. Lewis and J.M. Haviland-Jones, eds., third ed., pp. 88-101, 2008.
[7] W. James, "What Is an Emotion?" Mind, vol. 9, pp. 188-205, 1884.
[8] S. Schachter and J. Singer, "Cognitive, Social, and Physiological Determinants of Emotional State," Psychological Rev., vol. 69, pp. 379-399, 1962.
[9] W.B. Cannon, "The James-Lange Theory of Emotions: A Critical Examination and an Alternative Theory," Am. J. Psychology, vol. 39, pp. 106-124, 1927.
[10] J.L. Andreassi, "Heart Activity and Behavior: Stress, Emotions, Motivation, Personality, Social Factors, Brain Interactions, and Conditioning," Psychophysiology: Human Behavior and Physiological Response, pp. 241-249, 1995.
[11] J.T. Larsen, G.G. Berntson, K.M. Poehlmann, and T.A. Ito, and J.T. Cacioppo, "The Psychophysiology of Emotion," Handbook of Emotions, M. Lewis and J.M. Haviland-Jones, eds., third ed., pp. 173-191, 2008.
[12] S.D. Kreibig, "Autonomic Nervous System Activity in Emotion: A Review," Biological Psychology, vol. 84, no. 3, pp. 394-421, July 2010.
[13] R.W. Picard, E. Vyzas, and J. Healey, "Toward Machine Emotional Intelligence: Analysis of Affective Physiological State," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp. 1175-1191, Oct. 2001.
[14] J. Kim and E. André, "Emotion Recognition Based on Physiological Changes in Music Listening," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 12, pp. 2067-2083, Dec. 2008.
[15] K.H. Kim, S.W. Bang, and S.R. Kim, "Emotion Recognition System Using Short-Term Monitoring of Physiological Signals," Medical and Biological Eng. and Computing, vol. 42, pp. 1672-1687, 2004.
[16] R.A. Calvo and S. D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications," IEEE Trans. Affective Computing, vol. 1, no. 1, pp. 18-37, Jan. 2010.
[17] Z. Zeng, M. Pantic, G.l. Roisman, and T.S. Huang, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009.
[18] P. Philippot and G. Chapelle, "Respiratory Feedback in the Generation of Emotion," Cognition and Emotion, vol. 16, no. 5, pp. 605-627, Aug. 2002.
[19] F.A. Boiten, N.H. Frijda, and C.J.E. Wientjes, "Emotions and Respiratory Patterns: Review and Critical Analysis," Int'l J. Psychophysiology, vol. 17, pp. 103-128, 1994.
[20] C.L. Lisetti and F. Nasoz, "Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals," EURASIP J. Applied Signal Processing vol. 11, pp. 1672-1687, 2004.
[21] J. Rottenberg, R.D. Ray, and J.J. Gross, "Emotion-Elicitation Using Films," Handbook of Emotion Elicitation And Assessment, J.A. Coan, ed., Oxford Univ. Press, 2007.
[22] P. Gomez, S. Shafy, and B. Danuser, "Respiration, Metabolic Balance, and Attention in Affective Picture Processing," Biological Psychology, vol. 78, pp. 138-149, 2008.
[23] R.W. Picard, "Affective Computing: From Laughter to IEEE," IEEE Trans. Affective Computing, vol. 1, no. 1, pp. 11-17, Jan.-June 2010.
[24] F.A. Boiten, "The Effects of Emotional Behaviour on Components of the Respiratory Cycle," Biological Psychology, vol. 49, pp. 29-51, 1998.
[25] C.-K. Wu, P.-C. Chung, and C.-J. Wang, "Extracting Coherent Emotion Elicited Segments from Physiological Signals," Proc. IEEE Workshop Affective Computational Intelligence, 2011.
[26] P. Gomez, P. Zimmermann, S. Guttormsen-Schar, and B. Danuser, "Respiratory Responses Associated with Affective Processing of Film Stimuli," Biological Psychology, vol. 68, no. 3, pp. 223-235, Mar. 2005.
[27] P. Rainville, A. Bechara, N. Naqvi, and A.R. Damasio, "Basic Emotions Are Associated with Distinct Patterns of Cardiorespiratory Activity," Int'l J. Psychophysiology, vol. 61, pp. 5-18, 2006.
[28] M.A. Nicolaou, H. Gunes, and M. Pantic, "Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space," IEEE Trans. Affective Computing, vol. 2, no. 2, pp. 92-105, Apr.-June 2011.
[29] D. Michael and J. Houchin, "Automatic EEG Analysis: A Segmentation Procedure Based on the Autocorrelation Function," Electroencephalography and Clinical Neurophysiology, vol. 46, no. 2, pp. 232-235, Feb. 1979.
[30] L. Wong and W. Abdulla, "Time-Frequency Evaluation of Segmentation Methods for Neonatal EEG Signals" Proc. IEEE Int'l Conf. Eng. in Medicine and Biology Soc., pp. 1303-1306, 2006.
[31] U. Appel and A.v. Brandt, "A Comparative Analysis of Three Sequential Time-Series Segmentation Algorithms," Signal Processing, vol. 6, no. 1, pp. 45-60, Jan. 1984.
[32] D. Popivnov and A. Mineva, "Testing Procedures for Nonstationary and Non-Linearity in Physiological Signals," Math. Biosciences, vol. 157, nos. 1/2, pp. 303-320, 1999.
[33] E. Punskaya, C. Andrieu, A. Doucet, and W.J. Fitzgerald, "Bayesian Curve Fitting Using MCMC with Applications to Signal Segmentation," IEEE Trans. Signal Processing, vol. 50, no. 3, pp. 747-758, Mar. 2002.
[34] M. Staudacher, S. Telser, A. Amann, H. Hinterhuber, and M. Ritsch-Marte, "A New Method for Change-Point Detection Developed for On-Line Analysis of the Heart Beat Variability during Sleep," Physica A, vol. 349, nos. 3/4, pp. 682-596, Apr. 2005.
[35] C.K. Peng, S. Havlin, H.E. Stanley, and A.L. Goldberger, "Quantification of Scaling Exponents and Crossover Phenomena in Nonstationary Heartbeat Time-Series," Am. Inst. of Physics, vol. 5, pp. 82-87, 1995.
[36] A. Kaplan, J. Roschke, B. Darkhovsky, and J. Fell, "Macrostructural EEG Characterization Based on Nonparametric Change Point Segmentation: Application to Sleep Analysis," J. Neuroscience Methods, vol. 106, no. 1, pp. 81-90, Mar. 2001.
[37] E. Tan, "Film-Induced Affect as a Witness Emotion," Poetics, vol. 23, no. 1, pp. 7-32, 1995.
[38] L. Canini, S. Gilroy, M. Cavazza, R. Leonardi, and S. Benini, "Users' Response to Affective Film Content: A Narrative Perspective," Proc. Int'l Workshop Content-Based Multimedia Indexing, 2010.
[39] H. Peng, F. Long, and C. Ding, "Feature Selection Based on Mutual Information: Criteria of Max-Denendency, Max-Relevance, and Min-Redundancy," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1226-1238, Aug. 2005.
[40] P. Shaver, J. Schwartz, and D. Kirson, "Emotion Knowledge: Further Exploration of a Prototype Approach," J. Personality and Social Psychology, vol. 52, no. 6, pp. 1061-1086, 1987.
[41] S. Epstein, "Controversial Issues in Emotion Theory," Rev. of Personality and Social Psychology, vol. 5, pp. 64-88, 1984.
[42] D.A. Sauter, F. Eisner, P. Ekman, and S.K. Scott, "Cross-Cultural Recognition of Basic Emotions through Nonverbal Emotional Vocalizations," Proc. Nat'l Academy of Sciences USA, vol. 107, no. 6, pp. 2408-2412, Feb. 2010.
[43] B. Fehr and J.A. Russell, "Concept of Emotion Viewed from a Prototype Perspective," J. Experimental Psychology: General, vol. 113, pp. 464-486, 1984.
[44] I. Bretherton and M. Beeghly, "Talking about Internal States: The Acquisition of an Explicit Theory of Mind," Developmental Psychology, vol. 18, no. 6, pp. 906-912, 1982.
[45] J.A. Russell, "In Defense of a Prototype Approach to Emotion Concepts," J. Personality and Social Psychology, vol. 60, no. 1, pp. 37-47, 1991.
[46] A. Kappas, "Smile When You Read This, Whether You Like It or Not: Conceptual Challenges to Affect Detection," IEEE Trans. Affective Computing, vol. 1, no. 1, pp. 38-41, Jan. 2010.
[47] C. Peter and A. Herbon, "Emotion Representation and Physiology Assignments in Digital Systems," Interacting with Computers, vol. 18, no. 2, pp. 139-170, Mar. 2006.
[48] B.E. Brodsky and B.E. Darkhovsky, Nonparametric Methods in Change-Point Problems. Kluwer Academic Publishers, 1993.
[49] A. Denis and F. Cremoux, "Using the Entropy of Curves to Segment a Time or Spatial Series," Math. Geology, vol. 34, no. 8, pp. 899-914, Nov. 2002.
[50] M.D. Weber, L.M. Leemis, and R.K. Kincaid, "Minimum Kolmogorov-Smirnov Test Statistic Parameter Estimates," J. Statistical Computation and Simulation, vol. 76, no. 3, pp. 195-206, Mar. 2006.
[51] J. Richmann and J. Moorman, "Physiological Time-Series Analysis Using Approximate Entropy and Sample Entropy," Am. J. Physiology-Heart and Circulatory Physiology, vol. 278, no. 6, pp. H2039-H2049, June 2000.
[52] D. Caldirola, L. Bellodi, A. Caumo, G. Migliarese, and G. Perna, "Approximate Entropy of Respiratory Patterns in Panic Disorder," Am. J. Psychiatry, vol. 161, no. 1, pp. 79-87, 2004.
[53] C.D. Katsis, N. Katertsidis, G. Ganiatsas, and D.I. Fotiadis, "Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach," IEEE Trans. Systems, Man, and Cybernetics- Part A: Systems and Humans, vol. 38, no. 3, pp. 502-512, May 2008.
[54] G.M. Smith, Film Structure and the Emotion System. Cambridge Univ. Press, 2003.
[55] M. Soleymani, G. Chanel, J. Kierkels, and T. Pun, "Affective Ranking of Movie Scenes Using Physiological Signals and Content Analysis," Proc. ACM Workshop Multimedia Semantics, pp. 32-39, Oct. 2008.
[56] J. Listgarten, R.M. Neal, S.T. Roweis, and A. Emili, "Multiple Alignment of Continuous Time Series," Neural Information Processing Systems, vol. 17, 2005.
[57] R. JayaDevan, S.R. Kolhe, and P.M. Patil, "Dynamic Time Warping Based Static Hand Printed Signature Verification," J. Pattern Recognition Research, vol. 4, no. 1, pp. 52-65, 2009.
[58] I. Guyon and A. Elisseeff, "An Introduction to Variable and Feature Selection," J. Machine Learning Research, vol. 3, pp. 1152-1182, Mar. 2003.
[59] H. Peng mRMR, http://penglab.janelia.org/projmRMR/, 2011.
[60] D.L. Davies and D.W. Bouldin, "A Cluster Separation Measure," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 1, no. 2, pp. 224-227, Apr. 1979.
[61] P. Rainville, A. Bechara, N. Naqvi, A.R. Damasio, "Basic Emotions Are Associated with Distinct Patterns of Cardiorespiratory Activity," Int'l J. Psychophysiology, vol. 61, no. 1, pp. 5-18, July 2006.
[62] C.M. Bishop, "Probability Distributions," Pattern Recognition and Machine Learning, pp. 124-127, Springer, 2006.
[63] D. Cheng, G. Liu, and Y. Qiu, "Applications of Particle Swarm Optimization and K-Nearest Neighbors to Emotion Recognition from Physiological Signals," Proc. Int'l Conf. Computational Intelligence and Security, 2008.
[64] S. Chen, C.F.N. Cowan, and P.M. Grant, "Orthogonal Least Squares Learning Algorithm for Radial Basis Function Networks," IEEE Trans. Neural Networks, vol. 2, no. 2, pp. 302-309, Mar. 1991.
[65] N.H. Frijda, The Emotions, pp. 124-175. Cambridge Univ. Press, 1986.
[66] P. Lang, "The Emotion Probe: Studies of Motivation and Attention," Am. Psychologist, vol. 50, no. 5, pp. 372-385, 1995.
[67] C.C. Chang and C.J. Lin, "LIBSVM: A Library for Support Vector Machines Software," http://www.csie.ntu.edu.tw/~cjlinlibsvm, 2011.
17 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool