The Community for Technology Leaders
RSS Icon
Issue No.02 - April-June (2013 vol.4)
pp: 142-150
Ying Yang , University of Pittsburgh, Pittsburgh
Catherine Fairbairn , University of Pittsburgh, Pittsburgh
Jeffrey F. Cohn , University of Pittsburgh, Pittsburgh and Carnegie Mellon University, Pittsburgh
To investigate the relation between vocal prosody and change in depression severity over time, 57 participants from a clinical trial for treatment of depression were evaluated at seven-week intervals using a semistructured clinical interview for depression severity (Hamilton Rating Scale for Depression (HRSD)). All participants met criteria for major depressive disorder (MDD) at week one. Using both perceptual judgments by naive listeners and quantitative analyses of vocal timing and fundamental frequency, three hypotheses were tested: 1) Naive listeners can perceive the severity of depression from vocal recordings of depressed participants and interviewers. 2) Quantitative features of vocal prosody in depressed participants reveal change in symptom severity over the course of depression. 3) Interpersonal effects occur as well; such that vocal prosody in interviewers shows corresponding effects. These hypotheses were strongly supported. Together, participants' and interviewers' vocal prosody accounted for about 60 percent of variation in depression scores, and detected ordinal range of depression severity (low, mild, and moderate-to-severe) in 69 percent of cases (kappa $(= 0.53)$). These findings suggest that analysis of vocal prosody could be a powerful tool to assist in depression screening and monitoring over the course of depressive disorder and recovery.
Interviews, Switches, Timing, Speech, Atmospheric measurements, Particle measurements, Audio recording, hierarchical linear modeling (HLM), Prosody, switching pause, vocal fundamental frequency, depression, interpersonal influence
Ying Yang, Catherine Fairbairn, Jeffrey F. Cohn, "Detecting Depression Severity from Vocal Prosody", IEEE Transactions on Affective Computing, vol.4, no. 2, pp. 142-150, April-June 2013, doi:10.1109/T-AFFC.2012.38
[1] M. Alpert, E.R. Pouget, and R.R. Silva, "Reflections of Depression in Acoustic Measures of the Patient's Speech," J. Affective Disorders, vol. 66, no. 1, pp. 59-69, 2001.
[2] N. Ambady and R. Rosenthal, "Thin Slices of Expressive Behavior as Predictors of Interpersonal Consequences: A Meta-Analysis," Psychological Bull., vol. 111, no. 2, pp. 256-274, 1992.
[3] A.P. Association, "Diagnostic and Statistical Manual of Mental Disorders," Am. Psychiatric Assoc., 1994.
[4] B.A. Bettes, "Maternal Depression and Motherese: Temporal and International Features," Child Development, vol. 59, pp. 1089-1096, 1988.
[5] P. Boersma and D. Weenink, "Praat: Doing Phonetics by Computer," Univ. of Amsterdam, 2009.
[6] S.M. Boker and J.F. Cohn, "Real-Time Dissociation of Facial Appearance and Dynamics during Natural Conversation," Dynamic Faces: Insights from Experiments and Computation, H.H. Bülthoff, and M.A. Giese, eds., pp. 239-254, MIT Press, 2011.
[7] S.M. Boker, J.F. Cohn, B.J. Theobald, I. Matthews, J. Spies, and T. Brick, "Effects of Damping Head Movement and Facial Expression in Dyadic Conversation Using Real-Time Facial Expression Tracking and Synthesized Avatars," Philosophical Trans. B of the Royal Soc., vol. 364, pp. 3485-3495, 2009.
[8] K. Boudahmane, M. Manta, F. Antoine, S. Galliano, and C. Barras, Transcriberag, 2011.
[9] A.S. Bryk and S.W. Raudenbush, "Application of Hierarchical Linear Models to Assessing Change," Psychological Bull., vol. 101, pp. 147-158, 1987.
[10] S.B. Campbell and J.F. Cohn, "Prevalence and Correlates of Postpartum Depression in First-Time Mothers," J. Abnormal Psychology, vol. 100, no. 4, pp. 594-599, 1991.
[11] M. Cannizzaro, B. Harel, N. Reilly, P. Chappell, and P.J. Snyder, "Voice Acoustical Measurement of the Severity of Major Depression," Brain and Cognition, vol. 56, pp. 30-35, 2004.
[12] J.F. Cohn and S.B. Campbell, "Influence of Maternal Depression on Infant Affect Regulation," Developmental Perspectives on Depression, D. Cicchetti and S.L. Toth, eds., pp. 103-130, Univ. Rochester Press, 1992.
[13] J.C. Coyne, "Toward an Interactional Theory of Depression," Psychiatry, vol. 39, pp. 28-40, 1976.
[14] R.J. Davidson, ed., Anxiety, Depression, and Emotion, Oxford Univ., 2000.
[15] S. Duncan, "Some Signals and Rules for Taking Speaking Turns in Conversations," J. Personality & Social Psychology, vol. 23, no. 2, pp. 283-292, 1972.
[16] H. Ellgring and K.R. Scherer, "Vocal Indicators of Mood Change in Depression," J. Nonverbal Behavior, vol. 20, no. 2, pp. 83-110, 1996.
[17] R. Elliott, R. Zahn, J.F.W. Deakin, and I.M. Anderson, "Affective Cognition and Its Disruption in Mood Disorders," Neuropsychopharmacology, vol. 36, pp. 153-182, 2011.
[18] M.B. First, R.L. Spitzer, M. Gibbon, and J.B.W. Williams, Structured Clinical Interview for DSM-IV Axis I Disorders, SCID-I/P, Version 2.0 ed. Biometrics Research Department, New York State Psychiatric Institute-Patient Edition, 1995.
[19] J.L. Fleiss, Statistical Methods for Rates and Proportions. Wiley, 1981.
[20] J.C. Fournier, R.J. DeRubeis, S.D. Hollon, S. Dimidjian, J.D. Amsterdam, R.C. Shelton, and J. Fawcett, "Antidepressant Drug Effects and Depression Severity: A Patient-Level Meta-Analysis," J. Am. Medical Assoc., vol. 303, no. 1, pp. 47-53, 2010.
[21] D.J. France, "Acoustical Properties of Speech as Indicators of Depression and Suicidal Risk," IEEE Trans. Biomedical Eng., vol. 47, no. 7, pp. 829-837, July 2000.
[22] R.W. Frick, "Communicating Emotion: The Role of Prosodic Features," Psychological Bull., vol. 97, no. 3, pp. 412-429, 1985.
[23] M. Hamilton, "A Rating Scale for Depression," J. Neurology and Neurosurgery, vol. 23, pp. 56-61, 1960.
[24] E. Hatfield, J.T. Cacioppo, and R.L. Rapson, "Primitive Emotional Contagion," Emotion and Social Behavior, vol. 14 of Rev. Personality and Social Psychology, M.S. Clark, ed., pp. 151-177, Sage Publications, 1992.
[25] S.D. Hollon, M.E. Thase, and J.C. Markowitz, "Treatment and Prevention of Depression," Psychological Science in the Public Interest, vol. 3, no. 2, pp. 38-77, 2002.
[26] J. Jaffe, B. Beebe, S. Feldstein, C.L. Crown, and M. Jasnow, "Rhythms of Dialogue in Early Infancy," Monographs of the Soc. for Research in Child Development, vol. 66, pp. 1-8, 2001.
[27] J. Jaffe and S. Feldstein, Rhythms of Dialogue. Academic Press, 1970.
[28] P.N. Juslin and P. Laukka, "Communication of Emotions in Vocal Expression and Music Performance: Different Channels, Same Code?" Psychological Bull., vol. 129, pp. 770-814, 2003.
[29] F.N. Kerlinger, Foundations of Behavioral Research: Educational, Psychological and Sociological Inquiry. Holt, Rinehart and Winston, 1973.
[30] R. Kessler, W. Chiu, O. Demler, and E.E. Walters, "Prevalence, Severity, and Comorbidity of 12-Month dsm-iv Disorders in the National Comorbidity Survey Replication," Archives of General Psychiatry, vol. 62, pp. 617-627, 2005.
[31] R. Kotov, W. Gamez, F. Schmidt, and D. Watson, "Linking Personality Traits to Anxiety, Depressive, and Substance Use Disorders: A Meta-Analysis," Psychological Bull., vol. 136, no. 5, pp. 768-821, 2010.
[32] S. Kuny and H. Stassen, "Speaking Behavior and Voice Sound Characteristics in Depressive Patients during Recovery," J. Psychiatric Research, vol. 27, no. 3 pp. 289-307, 1993.
[33] E. Moore, M. Clements, J. Peifert, and L. Weisser, "Analysis of Prosodic Variation in Speech for Clinical Depression," Proc. IEEE 25th Ann. Int'l Conf. Eng. in Medicine and Biology Soc., 2003.
[34] J.C. Mundt, P.J. Snyder, M.S. Cannizzaro, K. Chappie, and D.S. Geraltsa, "Voice Acoustic Measures of Depression Severity and Treatment Response Collected via Interactive Voice Response (IVR) Technology," J. Neurolinguistics, vol. 20, pp. 50-64, 2007.
[35] P.M. Niedenthal, "Embodying Emotion," Science, vol. 316, pp. 1002-1005, 2007.
[36] W.H. Organization, The Global Burden of Disease: 2004 Update, World Health Organization, Geneva, Switzerland, 2008.
[37] A. Ozdas, R.G. Shiavi, S.E. Silverman, M.K. Silverman, and D.M. Wilkes, "Investigation of Vocal Jitter and Glottal Flow Spectrum as Possible Cues for Depression and Near-Term Suicidal Risk," IEEE Trans. Biomedical Eng., vol. 51, no. 9, pp. 1530-1540, Sept. 2004.
[38] A. Pentland, Honest Signals: How They Shape Our World. MIT Press, 2008.
[39] A. Pentland, "Kith and Kin," 2010.
[40] S.W. Raudenbush and A.S. Bryk, Hierarchical Linear Models: Applications and Data Analysis Methods, second ed., Sage. 2002.
[41] B.S. Reed, "Speech Rhythm Across Turn Transitions in Cross-Cultural Talk-In-Interaction," J. Pragmatics, vol. 42, no. 4, pp. 1037-1059, 2010.
[42] R. Rosenthal, "Conducting Judgment Studies," Handbook of Nonverbal Behavior Research Methods in the Affective Sciences, J.A. Harrigan, R. Rosenthal, and K.R. Scherer, eds., pp. 199-236, Oxford, 2005.
[43] K.R. Scherer and G. Ceschi, "Criteria for Emotion Recognition from Verbal and Nonverbal Expression: Studying Baggage Loss in the Airport," Personality and Social Psychology Bull., vol. 26, no. 3, pp. 327-339, 2000.
[44] B. Schuller, A. Batliner, S. Steidl, and D. Seppi, "Recognising Realistic Emotions and Affect in Speech: State of the Art and Lessons Learned from the First Challenge," Speech and Comm., vol. 53, nos. 9/10, pp. 1062-1087, 2010.
[45] Sphinx, "Cmu Sphinx: Open Source Toolkit for Speech Recognition," undated.
[46] A. Trevino, T. Quatieri, and N. Malyska, "Phonologically-Based Biomarkers for Major Depressive Disorder," EURASIP J. Advances in Signal Processing, vol. 42, article 2011, 2011.
[47] Z. Zeng, M. Pantic, G. Roisman, and T.S. Huang, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 31-58, Jan. 2009.
[48] A.J. Zlochower and J.F. Cohn, "Vocal Timing in Face-to-Face Interaction of Clinically Depressed and Nondepressed Mothers and Their 4-Month-Old Infants," Infant Behavior and Development, vol. 19, pp. 373-376, 1996.
21 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool