Subscribe
Issue No.03 - July-September (2012 vol.3)
pp: 349-365
Emilie Delaherche , University Pierre and Marie Curie, Paris
Mohamed Chetouani , University Pierre and Marie Curie, Paris
Ammar Mahdhaoui , Orange Labs R&D, Meylan Grenoble
Catherine Saint-Georges , University Pierre and Marie Curie, Paris
Sylvie Viaux , University Pierre and Marie Curie, Paris
David Cohen , University Pierre and Marie Curie, Paris
ABSTRACT
Synchrony refers to individuals' temporal coordination during social interactions. The analysis of this phenomenon is complex, requiring the perception and integration of multimodal communicative signals. The evaluation of synchrony has received multidisciplinary attention because of its role in early development, language learning, and social connection. Originally studied by developmental psychologists, synchrony has now captured the interest of researchers in such fields as social signal processing, robotics, and machine learning. This paper emphasizes the current questions asked by synchrony evaluation and the state-of-the-art related methods. First, we present definitions and functions of synchrony in youth and adulthood. Next, we review the noncomputational and computational approaches of annotating, evaluating, and modeling interactional synchrony. Finally, the current limitations and future research directions in the fields of developmental robotics, social robotics, and clinical studies are discussed.
INDEX TERMS
computational model, Synchrony evaluation, coordination
CITATION
Emilie Delaherche, Mohamed Chetouani, Ammar Mahdhaoui, Catherine Saint-Georges, Sylvie Viaux, David Cohen, "Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines", IEEE Transactions on Affective Computing, vol.3, no. 3, pp. 349-365, July-September 2012, doi:10.1109/T-AFFC.2012.12
REFERENCES
[1] A. Vinciarelli, M. Pantic, and H. Bourlard, “Social Signal Processing: Survey of an Emerging Domain,” Image and Vision Computing, vol. 27, no. 12, pp. 1743-1759, Nov. 2009.
[2] K. Prepin and C. Pelachaud, “Shared Understanding and Synchrony Emergence: Synchrony as An Indice of the Exchange of Meaning between Dialog Partners,” Proc. Int'l Conf. Agent and Artificial Intelligence, vol. 2, pp. 25-30, http://ken.prepin.free.fr/IMG/pdfSharedUnderstanding-Synchrony Emergence-final.pdf , Jan. 2011.
[3] K. Prepin and A. Revel, “Human-Machine Interaction as a Model of Machine-Machine Interaction : How to Make Machines Interact as Humans Do,” Advanced Robotics, vol. 21, no. 15, pp. 1709-1723, http://publi-etis.ensea.fr/2007PR07, Dec. 2007.
[4] J. Cassell, T. Bickmore, L. Campbell, H. Vilhj$\ddag$ lmsson, and H. Yan, “Conversation as a System Framework: Designing Embodied Conversational Agents,” Embodied Conversational Agents, pp. 29-63, MIT Press, 2000.
[5] A.N. Meltzoff, R. Brooks, A.P. Shon, and R.P.N. Rao, “‘Social’ Robots Are Psychological Agents for Infants: A Test of Gaze Following,” Neural Networks, vol. 23, nos. 8/9, pp. 966-972, http://www.sciencedirect.com/science/article/ B6T08-511K3S3-3/289e7637205f20da23591c8a274d5b341 , 2010.
[6] J. Nadel, I. Carchon, C. Kervella, D. Marcelli, and D. Réserbat-Plantey, “Expectancies for Social Contingency in 2-Month-Olds,” Developmental Science, vol. 2, no. 2, pp. 164-173, http://dx.doi.org/10.11111467-7687.00065 , 1999.
[7] H.H. Clark, Using Language, B.G. Blount ed., vol. 23, no. 4. Cambridge Univ. Press, http://www.amazon.com/Using- Language-Herbert-H-Clark/ dp0521567459, 1996.
[8] M.J. Pickering and S. Garrod, “Toward A Mechanistic Psychology of Dialogue,” Behavioral and Brain Sciences, vol. 27, no. 2, pp. 169-190, http://journals.cambridge.orgabstract_S0140525X04000056 , 2004.
[9] S. Garrod and M.J. Pickering, “Why Is Conversation So Easy?” Trends in Cognitive Sciences, vol. 8, no. 1, pp. 8-11, http://linkinghub.elsevier.com/retrieve/ piiS136466130300295X, 2004.
[10] S. Garrod and A. Anderson, “Saying What You Mean in Dialogue: A Study in Conceptual and Semantic Co-Ordination,” Cognition, vol. 27, no. 2, pp. 181-218, http://www.ncbi.nlm.nih.gov/pubmed3691025 , 1987.
[11] S. Garrod and A. Clark, “The Development of Dialogue Co-Ordination Skills in Schoolchildren,” Language and Cognitive Processes, vol. 8, no. 1, pp. 101-126, http://www.informaworld. com/10.108001690969308406950 , 1993.
[12] S.E. Brennan and H.H. Clark, “Conceptual Pacts and Lexical Choice in Conversation,” J. Experimental Psychology Learning Memory and Cognition, vol. 22, no. 6, pp. 1482-1493, 1996.
[13] H.P. Branigan, M.J. Pickering, and A.A. Cleland, “Syntactic Co-Ordination in Dialogue,” Cognition, vol. 75, no. 2, pp. B13-B25, http://dx.doi.org/10.1016S0010-0277(99)00081-5 , May 2000.
[14] H. Giles, J. Coupland, and N. Coupland, Accommodation Theory: Communication, Context, and Consequence, pp. 1-68. Cambridge Univ. Press, 1991.
[15] H. Giles, “Accent Mobility: A Model and Some Data,” Anthropological Linguistics, vol. 15, no. 2, pp. 87-105, 1973.
[16] R.Y. Bourhis and H. Giles, The Language of Intergroup Distinctiveness, pp. 119-134. Academic Press, http://scholar.google.comscholar?hl=en&btnG=Search&q=intitle:The+language+of+ intergroup+distinctiveness#0 , 1977.
[17] R.L. Street, “Speech Convergence and Speech Evaluation in Fact-Finding Interviews,” Human Comm. Research, vol. 11, no. 2, pp. 139-169, http://dx.doi.org/10.1111j.1468-2958.1984.tb00043.x , 1984.
[18] F.R. Bilous and R.M. Krauss, “Dominance and Accommodation in the Conversational Behaviours of Same- and Mixed-Gender Dyads,” Language and Comm., vol. 8, nos. 3/4, pp. 183-194, 1988.
[19] J.N. Cappella and S. Planalp, “Talk and Silence Sequences in Informal Conversations III: Interspeaker Influence,” Human Comm. Research, vol. 7, no. 2, pp. 117-132, http://dx.doi.org/10.1111j.1468-2958.1981.tb00564.x , 1981.
[20] W. Condon and W. Ogston, “A Segmentation of Behavior,” J. Psychiatric Research, vol. 5, pp. 221-235, 1967.
[21] F. Bernieri and R. Rosenthal, Interpersonal Coordination: Behavior Matching and Interactional Synchrony. Fundamentals of Nonverbal Behavior. Cambridge Univ. Press, 1991.
[22] J.L. Lakin and T.L. Chartrand, “Using Nonconscious Behavioral Mimicry to Create Affiliation and Rapport,” Psychological Science, vol. 14, no. 4, pp. 334-339, http://pss.sagepub.com/lookup/doi/10.1111 1467-9280.14481, 2003.
[23] T.L. Chartrand and A.N. Dalton, “Mimicry: Its Ubiquity, Importance and Functionality,” Oxford Handbook of Human Action, vol. 2, pp. 458-483, http://books.google.combooks?hl=en&lr= &id=zFP1AZYlNmcC&oi=fnd&pg=PA458&dq=Mimicry:+Its+ ubiquity,+importance,+and+functionality&ots=IVw1Ak_ICM& sig=yXINrSNo0VzZzJjoPOFK-R2mriI , 2009.
[24] X. Sun and A. Nijholt, “Multimodal Embodied Mimicry in Interaction,” Proc. Int'l Conf. Analysis of Verbal and Nonverbal Comm. and Enactment, A. Esposito, A. Vinciarelli, K. Vicsi, C. Pelachaud, and A. Nijholt, eds., pp. 147-153, 2011.
[25] T.L. Chartrand and J.A. Bargh, “The Chameleon Effect: The Perception-Behavior Link and Social Interaction,” J. Personality and Social Psychology, vol. 76, no. 6, pp. 893-910, 1999.
[26] G. Rizzolatti, L. Fadiga, V. Gallese, and L. Fogassi, “Premotor Cortex and the Recognition of Motor Actions,” Brain Research, vol. 3, no. 2, pp. 131-141, http://www.ncbi.nlm.nih.gov/pubmed8713554 , 1996.
[27] T. Chaminade and J. Decety, “A Common Framework for Perception and Action: Neuroimaging Evidence,” Behavioral and Brain Sciences, vol. 24, no. 5, pp. 879-882, 2001.
[28] L.O. Lundqvist, “Facial EMG Reactions to Facial Expressions: A Case of Facial Emotional Contagion?” Scandinavian J. Psychology, vol. 36, no. 2, pp. 130-141, http://www.pubmedcentral.nih.govarticlerender.fcgi?artid=2764895&tool=pmcentrez&rendertype= abstract , 1995.
[29] J. Cappella, Coding Mutual Adaptation in Dyadic Nonverbal Interaction, pp. 383-392. Lawrence Erlbaum, 2005.
[30] B. Newman and P. Newman, Development Through Life: A Psychosocial Approach, pp. 171-175. Cengage/Wadsworth, 2009.
[31] W. Condon and L. Sander, “Neonate Movement Is Synchronized with Adult Speech: Interactional Participation and Language Acquisition,” Science, vol. 183, pp. 99-101, 1974.
[32] A. Harrist and R. Waugh, “Dyadic Synchrony: Its Structure and Function in Children's Development,” Developmental Rev., vol. 22, no. 4, pp. 555-592, 2002.
[33] F. Ramseyer and W. Tschacher, “Synchrony: A Core Concept for a Constructivist Approach to Psychotherapy,” Constructivism. The Human Sciences., vol. 11, pp. 150-171, 2006.
[34] R. Feldman, “Infant-Mother and Infant-Father Synchrony: The Coregulation of Positive Arousal,” Infant Mental Health J., vol. 24, no. 1, pp. 1-23, 2003.
[35] A. Kendon, “Movement Coordination in Social Interaction: Some Examples Described,” Acta Psychologica, vol. 32, pp. 100-125, 1970.
[36] D.S. Schechter and E. Willheim, “Disturbances of Attachment and Parental Psychopathology in Early Childhood,” Child and Adolescent Psychiatric Clinics of North Am., vol. 18, no. 3, pp. 665-686, http://www.sciencedirect.com/science/article/ piiS1056499309000248, 2009.
[37] J. Kim-Cohen, G.A. Light, D.L. Braff, C.L.M. Caton, R.E. Drake, D.S. Hasin, P.E. Shrout, S. Samet, W.B. Schanzer, T.E. Moffitt, A. Taylor, S.J. Pawlby, A. Caspi, J.M. Hettema, C.A. Prescott, J.M. Myers, M.C. Neale, K.S. Kendler, M.R. Liebowitz, A.J. Gelenberg, and D. Munjack, “Maternal Depression and Children's Antisocial Behavior: Nature and Nurture Effects,” Archives of General Psychiatry, vol. 62, pp. 173-181, 2005.
[38] M. Rutter and T.G. O'Connor, “Are There Biological Programming Effects for Psychological Development? Findings from a Study of Romanian Adoptees,” Developmental Psychology, vol. 40, no. 1, pp. 81-94, http://www.ncbi.nlm.nih.gov/pubmed14700466 , 2004.
[39] P. Perruchet and S. Pacton, “Implicit Learning and Statistical Learning: One Phenomenon, Two Approaches,” Trends in Cognitive Sciences, vol. 10, no. 5, pp. 233-238, 2006.
[40] J.R. Saffran, R.N. Aslin, and E.L. Newport, “Statistical Learning by 8-Month-Old Infants,” Science, vol. 274, pp. 1927-1928, Dec. 1996.
[41] P.K. Kuhl, “Is Speech Learning ‘Gated’ by the Social Brain?” Developmental Science, vol. 10, no. 1, pp. 110-120, http://dx.doi.org/10.1111j.1467-7687.2007.00572.x , 2007.
[42] P.K. Kuhl, “Early Language Acquisition: Cracking the Speech Code,” Nature Rev. Neuroscience, vol. 5, no. 11, pp. 831-843, http://dx.doi.org/10.1038nrn1533, Nov. 2004.
[43] P.K. Kuhl, J.E. Andruski, I.A. Chistovich, L.A. Chistovich, E.V. Kozhevnikova, V.L. Ryskina, E.I. Stolyarova, U. Sundberg, and F. Lacerda, “Cross-Language Analysis of Phonetic Units in Language Addressed to Infants,” Science, vol. 277, pp. 684-686, 1997.
[44] P.K. Kuhl, “A New View of Language Acquisition,” Proc. Nat'l Academy of Sciences USA, vol. 97, pp. 11850-11857, 2000.
[45] M.H. Goldstein, A.P. King, and M.J. West, “Social Interaction Shapes Babbling: Testing Parallels between Birdsong and Speech,” Proc. Nat'l Academy of Sciences USA, vol. 100, no. 13, pp. 8030-8035, http://www.pubmedcentral.nih.govarticlerender.fcgi?artid= 164707&tool=pmcentrez&rendertype=abstract , 2003.
[46] J. Piaget, La Formation Du Symbole Chez l'enfant: Imitation, Jeu et Reve, Image et Représentation, series Actualités Pédagogiques et Psychologiques. Delachaux et Niestlé, http://books.google.frbooks?id=KYq3AAAAIAAJ , 1976.
[47] A.N. Meltzoff and M.K. Moore, “Imitation et Développement Humain: Les Premiers Temps de la Vie,” Terrain, vol. 44, pp. 71-90, http://terrain.revues.org2455, 2005.
[48] M. Lafrance, Posture Mirroring and Rapport, pp. 279-298. Human Sciences Press, 1982.
[49] L.K. Miles, L.K. Nind, and C.N. Macrae, “The Rhythm of Rapport: Interpersonal Synchrony and Social Perception,” J. Experimental Social Psychology, vol. 45, no. 3, pp. 585-589, http://www. sciencedirect.com/science/article/ piiS0022103109000328, 2009.
[50] D. Lakens, “Movement Synchrony and Perceived Entitativity,” J. Experimental Social Psychology, vol. 46, no. 5, pp. 701-708, http://www.sciencedirect.com/science/article/ B6WJB-4YRHD1M-1/2dd9c76660bd64b6a00d4bbec2b08e1bd , 2010.
[51] S.S. Wiltermuth and C. Heath, “Synchrony and Cooperation,” Psychological Science, vol. 20, no. 1, pp. 1-5, 2009.
[52] C.N. Macrae, O.K. Duffy, L.K. Miles, and J. Lawrence, “A Case of Hand Waving: Action Synchrony and Person Perception,” Cognition, vol. 109, no. 1, pp. 152-156, http://www.science direct.com/science/article/ piiS0010027708001686, 2008.
[53] L.K. Miles, L.K. Nind, Z. Henderson, and C.N. Macrae, “Moving Memories: Behavioral Synchrony and Memory for Self and Others,” J. Experimental Social Psychology, vol. 46, no. 2, pp. 457-460, http://www.sciencedirect.com/science/article/ piiS0022103109003126, 2010.
[54] F. Ramseyer and W. Tschacher, “Nonverbal Synchrony in Psychotherapy: Coordinated Body Movement Reflects Relationship Quality and Outcome,” J. Consulting and Clinical Psychology, vol. 79, no. 3, pp. 284-295, http://www.sciencedirect.com/science/article/ piiS0022006X11600314, 2011.
[55] A.L. Bouhuys and M.M. Sam, “Lack of Coordination of Nonverbal Behaviour Between Patients and Interviewers as a Potential Risk Factor to Depression Recurrence: Vulnerability Accumulation in Depression,” J. Affective Disorders, vol. 57, nos. 1-3, pp. 189-200, http://www.sciencedirect.com/science/article/ piiS0165032799000932, 2000.
[56] G. Bird, J. Leighton, C. Press, and C. Heyes, “Intact Automatic Imitation of Human and Robot Actions in Autism Spectrum Disorders,” Proc. Royal Soc. B Biological Sciences, vol. 274, no. 1628, pp. 3027-3031, http://www.ncbi.nlm.nih.gov/pubmed17911053 , 2007.
[57] M. Kipp, “Spatiotemporal Coding in Anvil,” Proc. Sixth Int'l Conf. Language Resources and Evaluation, 2008.
[58] J. Cappella, “Behavioral and Judged Coordination in Adult Informal Social Interactions: Vocal and Kinesic Indicators,” Personality and Social Psychology, vol. 72, pp. 119-131, 1997.
[59] C. Trevarthen and S. Daniel, “Disorganized Rhythm and Synchrony: Early Signs of Autism and Rett Syndrome,” Brain and Development, vol. 27, no. 1, pp. S25-S34, 2005.
[60] J. Cappella, A Procedure to Measure Interactional Synchrony in the Context of Satisfied and Dissatisfied Couples' Communication, pp. 199-208. Lawrence Erlbaum, 2005.
[61] C. Saint-Georges, R. Cassel, D. Cohen, M. Chetouani, M.-C. Laznik, S. Maestro, and F. Muratori, “What Studies of Family Home Movies Can Teach Us about Autistic Infants: A Literature Review,” Research in Autism Spectrum Disorders, vol. 4, no. 3, pp. 355-366, 2010.
[62] F. Bernieri, J. Reznick, and R. Rosenthal, “Synchrony, Pseudo Synchrony, and Dissynchrony: Measuring the Entrainment Process in Mother-Infant inTeractions,” J. Personality and Social Psychology, vol. 54, no. 2, pp. 243-253, 1988.
[63] C. Trevarthen and K.J. Aitken, “Infant Intersubjectivity: Research, Theory, and Clinical Applications,” The J. Child Psychology and Psychiatry and Allied Disciplines, vol. 42, no. 1, pp. 3-48, 2001.
[64] R. Feldman, “Parent-Infant Synchrony and the Construction of Shared Timing; Physiological Precursors, Developmental Outcomes, and Risk Conditions,” J. Child Psychology and Psychiatry and Allied Disciplines, vol. 48, nos. 3/4, pp. 329-354, http://www. ncbi.nlm.nih.gov/pubmed17355401 , 2007.
[65] C. Trevarthen, Communication and Cooperation in Early Infancy: A Description of Primary Intersubjectivity, pp. 321-347. Cambridge Univ. Press, 1979.
[66] R. Feldman, M. Keren, O. Gross-Rozval, and S. Tyano, “Mother-Child Touch Patterns in Infant Feeding Disorders: Relation to Maternal, Child, and Environmental Factors,” J. Am. Academy of Child and Adolescent Psychiatry, vol. 43, no. 9, pp. 1089-1097, http://www.sciencedirect.com/science/article/ piiS089085670961443X, 2004.
[67] D. Dollberg, R. Feldman, and M. Keren, “Maternal Representations, Infant Psychiatric Status, and Mother-Child Relationship in Clinic-Referred and Non-Referred Infants,” European Child and Adolescent Psychiatry, vol. 19, pp. 25-36, http://dx.doi.org/10. 1007s00787-009-0036-5 , 2010.
[68] D. Silberstein, R. Feldman, J.M. Gardner, B.Z. Karmel, J. Kuint, and R. Geva, “The Mother-Infant Feeding Relationship across the First Year and the Development of Feeding Difficulties in Low-Risk Premature Infants,” Infancy, vol. 14, no. 5, pp. 501-525, http://www.tandfonline.com/doi/abs/10.1080 15250000903144173, 2009.
[69] R. Feldman and A.I. Eidelman, “Direct and Indirect Effects of Breast Milk on the Neurobehavioral and Cognitive Development of Premature Infants,” Developmental Psychobiology, vol. 43, no. 2, pp. 109-119, http://dx.doi.org/10.1002dev.10126, 2003.
[70] R. Feldman, A. Weller, L. Sirota, and A.I. Eidelman, “Skin-to-Skin Contact (Kangaroo Care) Promotes Self-Regulation in Premature Infants: Sleep-Wake Cyclicity, Arousal Modulation, and Sustained Exploration,” Developmental Psychology, vol. 38, no. 2, pp. 194-207, http://doi.apa.org/getdoi.cfm?doi=10.1037 0012-1649.38.2.194, 2002.
[71] R. Feldman, A.I. Eidelman, and N. Rotenberg, “Parenting Stress, Infant Emotion Regulation, Maternal Sensitivity, and the Cognitive Development of Triplets: A Model for Parent and Child Influences in a Unique Ecology,” Child Development, vol. 75, no. 6, pp. 1774-91, http://www.ncbi.nlm.nih.gov/pubmed15566379 , 2004.
[72] G. Chittaranjan, O. Aran, and D. Gatica-Perez, “Inferring Truth from Multiple Annotators for Social Interaction Analysis,” Proc. Neural Information Processing Systems Workshop Modeling Human Comm. Dynamics, p. 4, 2011.
[73] U. Altmann, “Studying Movement Synchrony Using Time Series and Regression Models,” I.A. Esposito R. Hoffmann, S. Hübler, and B.W., eds., p. 23, 2011.
[74] R. Levitan, A. Gravano, and J. Hirschberg, “Entrainment in Speech Preceding Backchannels,” Proc. Ann. Meeting Assoc. for Computational Linguistics, pp. 113-117, http://publicaciones.dc.uba.ar/Publications/ 2011LGH11, 2011.
[75] Sˇtefan Benˇusˇ, A. Gravano, and J. Hirschberg, “Pragmatic Aspects of Temporal Accommodation in Turn-Taking,” J. Pragmatics, vol. 43, no. 12, pp. 3001-3027, http://www.sciencedirect.com/science/article/ piiS0378216611001469, 2011.
[76] J.C. Acosta and N.G. Ward, “Achieving Rapport with Turn-by-Turn, User-Responsive Emotional Coloring,” Speech Comm., vol. 53, pp. 1137-1148, http://dx.doi.org/10.1016j.specom.2010. 11.006 , Nov. 2011.
[77] E. Delaherche and M. Chetouani, “Multimodal Coordination: Exploring Relevant Features and Measures,” Proc. ACM Multimedia Second Int'l Workshop Social Signal Processing, 2010.
[78] A.V. Barbosa, E. Vatikiotis-Bateson, M. Oberg, and R.-M. Déchaine, “An Instantaneous Correlation Algorithm for Assessing Intra and Inter Subject Coordination During Communicative Behavior,” Proc. Neural Information Processing Systems Workshop Modeling Human Comm. Dynamics, 2010.
[79] O. Oullier, G.C. de Guzman, K.J. Jantzen, J.A. Scott Kelso, and J. Lagarde, “Social Coordination Dynamics: Measuring Human Bonding,” Social Neuroscience, vol. 3, no. 2, pp. 178-192, http://dx.doi.org10.1371%2Fjournal.pcbi.1000029 , 2008.
[80] D.C. Richardson and R. Dale, “Looking to Understand: The Coupling between Speakers' and Listeners' Eye Movements and Its Relationship to Discourse Comprehension,” Cognitive Science, vol. 29, no. 6, pp. 1045-1060, 2005.
[81] R. Schmidt, B. O'Brien, and R. Sysko, “Self-Organization of Between-Persons Cooperative Tasks and Possible Applications to Sport,” Int'l J. Sport Psychology, vol. 30, pp. 558-579, 1999.
[82] R. Schmidt, C. Carello, and M. Turvey, “Phase Transitions and Critical Fluctuations in the Visual Coordination of Rhythmic Movements Between People,” J. Experimental Psychology: Human Perception and Performance, vol. 16, no. 2, pp. 227-247, http://www.sciencedirect.com/science/article/ piiS0096152302012282, 1990.
[83] K. Shockley, M.-V. Santana, and C.A. Fowler, “Mutual Interpersonal Postural Constraints Are Involved in Cooperative Conversation,” J. Experimental Psychology: Human Perception and Performance, vol. 29, pp. 326-332, 2003.
[84] D. Richardson, R. Dale, and K. Shockley, Synchrony and Swing in Conversation: Coordination, Temporal Dynamics, and Communication. Oxford Univ. Press, 2008.
[85] K.D. Shockley, A.A. Baker, M.J. Richardson, and C.A. Fowler, “Verbal Constraints on Interpersonal Postural Coordination,” J. Experimental Psychology: Human Perception and Performance, vol. 33, pp. 201-208, 2007.
[86] K.T. Ashenfelter, S.M. Boker, J.R. Waddell, and N. Vitanov, “Spatiotemporal Symmetry and Multifractal Structure of Head Movements during Dyadic Conversation,” J. Experimental Psychology Human Perception and Performance, vol. 35, no. 4, pp. 1072-1091, http://www.biomedsearch.com/nih/Spatiotemporal-symmetry-multifractal-structure-head 19653750.html , 2009.
[87] S.M. Boker, M. Xu, J.L. Rotondo, and K. King, “Windowed Cross-Correlation and Peak Picking for the Analysis of Variability in the Association between Behavioral Time Series,” Psychological Methods, vol. 7, no. 3, pp. 338-355, http://www.sciencedirect.com/science/article/ B6WYR-474225D-6/21ef22d99bbf93112bc2c1089637a2739 , 2002.
[88] R. Rienks, R. Poppe, and D. Heylen, “Differences in Head Orientation Behavior for Speakers and Listeners: An Experiment in a Virtual Environment,” ACM Trans. Applied Perception, vol. 7, pp. 2:1-2:13, http://doi.acm.org/10.11451658349.1658351 , Jan. 2010.
[89] N. Campbell, “Multimodal Processing of Discourse Information; the Effect of Synchrony,” Proc. Second Int'l Symp. Universal Comm., pp. 12-15, 2008.
[90] N. Campbell, “Automatic Detection of Participant Status and Topic Changes in Natural Spoken Dialogues,” 2008.
[91] N. Campbell, “An Audio-Visual Approach to Measuring Discourse Synchrony in Multimodal Conversation Data,” Proc. Interspeech, pp. 2159-2162, 2009.
[92] G. Varni, G. Volpe, and A. Camurri, “A System for Real-Time Multimodal Analysis of Nonverbal Affective Social Interaction in User-Centric Media,” IEEE Trans. Multimedia, vol. 12, no. 6, pp. 576 -590, Oct. 2010.
[93] G. Varni, A. Camurri, P. Coletta, and G. Volpe, “Emotional Entrainment in Music Performance,” Proc. IEEE Eighth Int'l Conf. Automatic Face Gesture Recognition, pp. 1 -5, Sept. 2008.
[94] A. Camurri, G. Varni, and G. Volpe, “Measuring Entrainment in Small Groups of Musicians,” Proc. Third Int'l Conf. Affective Computing and Intelligent Interaction and Workshops, pp. 1-4, Sept. 2009.
[95] G. Varni, A. Camurri, P. Coletta, and G. Volpe, “Toward a Real-Time Automated Measure of Empathy and Dominance,” Proc. Int'l Conf. Computational Science and Eng., pp. 843-848, 2009.
[96] S.M. Boker and J.L. Rotondo, Symmetry Building and Symmetry Breaking in Synchronized Movement, vol. 42, pp. 163-171. John Benjamins Publishing Company, 2002.
[97] E. Keller and W. Tschacher, “Prosodic and Gestural Expression of Interactional Agreement,” Proc. Int'l Conf. Verbal and Nonverbal Comm. Behaviours COST 2102 Workshop, pp. 85-98, 2007.
[98] D. Reidsma, A. Nijholt, W. Tschacher, and F. Ramseyer, “Measuring Multimodal Synchrony for Human-Computer Interaction,” Proc. Int'l Conf. Cyberworlds, Oct. 2010.
[99] F. Ramseyer and W. Tschacher, “Nonverbal Synchrony or Random Coincidence? How to Tell the Difference,” Proc. Second Int'l Conf. Development of Multimodal Interfaces: Active Listening and Synchrony, A. Esposito et al., eds., pp. 182-196, 2010.
[100] C. Nagaoka and M. Komori, “Body Movement Synchrony in Psychotherapeutic Counseling: A Study Using the Video-Based Quantification Method,” IEICE Trans. Information and Systems, vol. E91-D, no. 6, pp. 1634-1640, http://hdl.handle.net/243385345, 2008.
[101] M. Komori, K. Maeda, and C. Nagaoka, “A Video-Based Quantification Method of Body Movement Synchrony: An Application for Dialogue in Counseling,” Japanese J. Interpersonal and Social Psychology, pp. 182-196, 2007.
[102] X. Sun, K.P. Truong, M. Pantic, and A. Nijholt, “Towards Visual and Vocal Mimicry Recognition in Human-Human Interactions,” Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, special session on social signal processing, E. Tunstel, S. Nahavandi, and A. Stoica, eds., pp. 367-373, http://doc.utwente. nl78613/, Nov. 2011.
[103] P. Amazeen, R. Schmidt, and M. Turvey, “Frequency Detuning of the Phase Entrainment Dynamics of Visually Coupled Rhythmic Movements,” Biological Cybernetics, vol. 72, pp. 511-518, http://dx.doi.org/10.1007BF00199893, 1995.
[104] R.C. Schmidt, M. Bienvenu, P.A. Fitzpatrick, and P.G. Amazeen, “A Comparison of Intra- and Interpersonal Interlimb Coordination: Coordination Breakdowns and Coupling Strength,” J. Experimental Psychology: Human Perception and Performance, vol. 24, no. 3, pp. 884-900, http://www.sciencedirect.com/science/article/ piiS009615230200411X, 1998.
[105] R.C. Schmidt, N. Christianson, C. Carello, and R. Baron, “Effects of Social and Physical Variables on Between-Person Visual Coordination,” Ecological Psychology, vol. 6, pp. 159-183, 1994.
[106] R.C. Schmidt and B. O'Brien, “Evaluating the Dynamics of Unintended Interpersonal Coordination,” Ecological Psychology, vol. 9, no. 3, pp. 189-206, http://dx.doi.org/10.1207s15326969eco0903_2 , 1997.
[107] M.J. Richardson, K.L. Marsh, R.W. Isenhower, J.R. Goodman, and R. Schmidt, “Rocking Together: Dynamics of Intentional and Unintentional Interpersonal Coordination,” Human Movement Science, vol. 26, no. 6, pp. 867-891, http://www.sciencedirect. com/science/article/ B6V8T-4PJD9H6-1/2f8c6d9dcb7ddf3e0430 ae264ea1f2944 , 2007.
[108] X. Sun, K. Truong, A. Nijholt, and M. Pantic, “Automatic Visual Mimicry Expression Analysis in Interpersonal Interaction,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition Workshops for Human Behaviour Analysis, pp. 40-46, June 2011.
[109] J. Webber and C.L. Zbilut, “Dynamical Assessment of Physiological Systems and States Using Recurrence Plot Strategies,” J. Applied Physiology, vol. 76, pp. 965-973, 1994.
[110] O. Oullier and J.A. Kelso, Social Coordination from the Perspective of Coordination Dynamics. Springer-Verlag, 2009.
[111] R. Schmidt and M. Turvey, “Phase-Entrainment Dynamics of Visually Coupled Rhythmic Movements,” Biological Cybernetics, vol. 70, pp. 369-376, http://dx.doi.org/10.1007BF00200334, 1994.
[112] J. Issartel, L. Marin, and M. Cadopi, “Unintended Interpersonal Co-Ordination: Can We March to the Beat of Our Own Drum?” Neuroscience Letters, vol. 411, no. 3, pp. 174-179, http://www.sciencedirect.com/science/article/ piiS0304394006009505. 2007.
[113] A. Fogel, “Two Principles of Communication: Co-Regulation and Framing,” New Perspectives in Early Communicative Development, series Int'l Library of Psychology, Routledge, http://books.google.frbooks?id=mXYOAAAAQAAJ , 1993.
[114] D.M. Messinger, P. Ruvolo, N.V. Ekas, and A. Fogel, “Applying Machine Learning to Infant Interaction: The Development Is in the Details,” Neural Networks, vol. 23, nos. 8/9, pp. 1004-1016, http://www.sciencedirect.com/science/article/ piiS0893608010001590, 2010.
[115] M.S. Magnusson, “Discovering Hidden Time Patterns in Behavior: T-Patterns and Their Detection,” Behaviour Research Methods, Instruments and Computers, vol. 32, no. 1, pp. 93-110, http://view.ncbi.nlm.nih.gov/pubmed10758668 , Feb. 2000.
[116] M. Plumet and C. Tardif, “Understanding the Functioning of Social Interaction with Autistic Children,” http:/www. emergingcommunication.com, 2005.
[117] K. Sakaguchi and G.K. Jonsson, “Initial Interpersonal Attraction between Mixed-Sex Dyad and Movement Synchrony,” The Hidden Structure of Interaction from Neurons to Culture Patterns, pp. 108-120, IOS Press, 2005.
[118] K. Grammer, K.B. Kruck, and M.S. Magnusson, “The Courtship Dance: Patterns of Nonverbal Synchronization in Opposite-Sex Encounters,” J. Nonverbal Behavior, vol. 22, no. 1, pp. 3-29, www.springerlink.com/indexh62v413694458276.pdf , 1998.
[119] M.T. Anguera, “Analysis of Symmetry/Asymmetry in Social Interaction,” The Hidden Structure of Interaction from Neurons to Culture Patterns, http:/www.emergingcommunication.com, pp. 52-70, IOS Press, 2005.
[120] C. Saint-Georges, A. Mahdhaoui, M. Chetouani, R.S. Cassel, M.-C. Laznik, F. Apicella, P. Muratori, S. Maestro, F. Muratori, and D. Cohen, “Do Parents Recognize Autistic Deviant Behavior Long Before Diagnosis? Taking into Account Interaction Using Computational Methods,” PLoS ONE, vol. 6, no. 7, p. e22393, http://dx.doi.org10.1371%2Fjournal.pone.0022393 , 2011.
[121] A. Mahdhaoui and M. Chetouani, “Understanding Parent-Infant Behaviors Using Non-Negative Matrix Factorization,” Proc. Third COST 2102 Int'l Training School Conf. Toward Autonomous, Adaptive, and Context-Aware Multimodal Interfaces: Theoretical and Practical Issues, pp. 436-447, http://portal.acm.orgcitation.cfm?id= 1950280.1950320 , 2011.
[122] P. Paatero and U. Tapper, “Positive Matrix Factorization: A Non-Negative Factor Model with Optimal Utilization of Error Estimates of Data Values,” Environmetrics, vol. 5, no. 2, pp. 111-126, http://dx.doi.org/10.1002env.3170050203, 1994.
[123] A. Strehl and J. Ghosh, “Cluster Ensembles a Knowledge Reuse Framework for Combining Multiple Partitions,” J. Machine Learning Research, vol. 3, pp. 583-617, 2002.
[124] D. Ozkan, K. Sagae, and L.-p. Morency, “Latent Mixture of Discriminative Experts for Multimodal Prediction Modeling,” Proc. 23rd Int'l Conf. Computational Linguistics, pp. 860-868, http://www.aclweb.org/anthologyC10-1097, 2010.
[125] D. Neiberg and J. Gustafson, “Predicting Speaker Changes and Listener Responses with and without Eye-Contact,” Proc. 12th Ann. Conf. Int'l Speech Comm. Assoc., pp. 1565-1568, 2011.
[126] L. Huang, L.-P. Morency, and J. Gratch, “A Multimodal End-of-Turn Prediction Model: Learning from Parasocial Consensus Sampling,” Proc. 10th Int'l Conf. Autonomous Agents and Multiagent Systems, vol. 3, pp. 1289-1290, http://dl.acm.orgcitation.cfm?id= 2034396.2034530 , 2011.
[127] C.-C. Lee and S. Narayanan, “Predicting Interruptions in Dyadic Spoken Interactions,” Proc. IEEE Int'l Conf. Acoustics Speech and Signal Processing, pp. 5250-5253, 2010.
[128] N.G. Ward, O. Fuentes, and A. Vega, “Dialog Prediction for a General Model of Turn-Taking,” Proc. Interspeech '10, pp. 2662-2665, 2010.
[129] K.R. Thorisson, Natural Turn-Taking Needs No Manual: Computational Theory and Model, from Perception to Action, pp. 173-207. Kluwer Acedemic Publishers, http://xenia.media.mit.edu/% 7Ekris/ftpCompModTurnTak.pdf , 2002.
[130] N. Cathcart, J. Carletta, and E. Klein, “A Shallow Model of Backchannel Continuers in Spoken Dialogue,” Proc. 10th Conf. European Chapter of the Assoc. for Computational Linguistics, vol. 1, pp. 51-58, http://dx.doi.org/10.31151067807. 1067816 , 2003.
[131] A. Gravano and J. Hirschberg, “Backchannel-Inviting Cues in Task-Oriented Dialogue,” Proc. Interspeech, pp. 1019-1022, http://dblp.uni-trier.de/db/conf/interspeech interspeech2009.html# GravanoH09, 2009.
[132] L.-P. Morency, I. Kok, and J. Gratch, “Predicting Listener Backchannels: A Probabilistic Multimodal Approach,” Proc. Eighth Int'l Conf. Intelligent Virtual Agents, pp. 176-190, http://dx.doi.org/10.1007978-3-540-85483-8_18 , 2008.
[133] J. Nadel, K. Prepin, and M. Okanda, “Experiencing Contingency and Agency: First Step Toward Self-Understanding in Making a Mind?” Interaction Studies, vol. 6, no. 3, pp. 447-462, http://openurl.ingenta.com/contentxref?genre=article&issn=1572-0373&volume=6&issue=3&spage=447 , 2005.
[134] E. Delaherche and M. Chetouani, “Characterization of Coordination in An Imitation Task: Human Evaluation and Automatically Computable Cues” Proc. 13th Int'l Conf. Multimodal Interaction, 2011.
[135] X. Sun, J. Lichtenauer, M. Valstar, A. Nijholt, and M. Pantic, “A Multimodal Database for Mimicry Analysis,” Proc. Fourth Int'l Conf. Affective Computing and Intelligent Interaction, Part I, S. D'Mello, A. Graesser, B. Schuller, and J.-C. Martin, eds., pp. 367-376, http://doc.utwente.nl78299/, Oct. 2011.
[136] A.N. Meltzoff, P.K. Kuhl, J. Movellan, and T.J. Sejnowski, “Foundations for a New Science of Learning,” Science, vol. 325, no. 5938, pp. 284-288, http://www.sciencemag.org/cgi/content/full/ 325/5938284, 2009.
[137] A. Cangelosi, G. Metta, G. Sagerer, S. Nolfi, C. Nehaniv, K. Fischer, J. Tani, T. Belpaeme, G. Sandini, F. Nori, L. Fadiga, B. Wrede, K. Rohlfing, E. Tuci, K. Dautenhahn, J. Saunders, and A. Zeschel, “Integration of Action and Language Knowledge: A Roadmap for Developmental Robotics,” IEEE Trans. Autonomous Mental Development, vol. 2, no. 3, pp. 167-195, http://dx.doi.org/10.1109TAMD.2010.2053034 , Sept. 2010.
[138] K. Prepin and P. Gaussier, “How an Agent Can Detect and Use Synchrony Parameter of Its Own Interaction with a Human?” Proc. Second Int'l Conf. Development of Multimodal Interfaces: Active Listening and Synchrony, A. Esposito et al., eds., pp. 50-65, 2010.
[139] J. Gratch, N. Wang, J. Gerten, E. Fast, and R. Duffy, “Creating Rapport with Virtual Agents,” Proc. Seventh Int'l Conf. Intelligent Virtual Agents, pp. 125-138, http://dx.doi.org/10.1007978-3-540-74997-4_12 , 2007.
[140] M. Michalowski, R. Simmons, and H. Kozima, “Rhythmic Attention in Child-Robot Dance Play,” Proc. IEEE 18th Int'l Symp. Robot and Human Interactive Comm., 2009.
[141] H. Kozima, M. Michalowski, and C. Nakagawa, “Keepon,” Int'l J. Social Robotics, vol. 1, pp. 3-18, http://dx.doi.org/10.1007s12369-008-0009-8 , 2009.
[142] J.J. Diehl, L.M. Schmitt, M. Villano, and C.R. Crowell, “The Clinical Use of Robots for Individuals with Autism Spectrum Disorders: A Critical Review,” Research in Autism Spectrum Disorders, vol. 6, pp. 249-262, http://www.sciencedirect.com/science/article/ piiS1750946711000894, 2011.
[143] M. Milgram and H. Atlan, “Probabilistic Automata as a Model for Epigenesis of Cellular Networks,” J. Theoretical Biology, vol. 103, pp. 523-547, 1983.
[144] R.P.N. Rao, A.P. Shon, and A.N. Meltzoff, “A Bayesian Model of Imitation in Infants and Robots,” Imitation and Social Learning in Robots Humans and Animals, pp. 217-247, Cambridge Univ. Press, 2007.
[145] D. Cohen, “Probabilistic Epigenesis: An Alternative Causal Model for Conduct Disorders in Children and Adolescents,” Neuroscience & Biobehavioral Rev., vol. 34, no. 1, pp. 119-129, http://www.sciencedirect.com/science/article/ piiS0149763409001079, 2010.
[146] C.-c. Lee, A. Katsamanis, M.P. Black, B.R. Baucom, P.G. Georgiou, and S.S. Narayanan, “An Analysis of PCA-Based Vocal Entrainment Measures in Married Couples' Affective Spoken Interactions,” Proc. InterSpeech, pp. 3101-3104, 2011.
[147] J.F. Cohn, “Advances in Behavioral Science Using Automated Facial Image Analysis and Synthesis,” IEEE Signal Processing Magazine, vol. 27, no. 6, pp. 128-133, http://ieeexplore.ieee.org/lpdocs/epic03 wrapper.htm?arnumber=5563099, Nov. 2010.
[148] D. Stern, A. Lazartigues, and D. Pérard, “Le Monde Interpersonnel du Nourrisson: Une Perspective Psychanalytique et Développementale,” ser. Le Fil rouge, Section 2, Psychanalyse et Psychiatrie de l'Enfant, Presses universitaires de France, 2003.
[149] A. Tartaro and J. Cassell, “Playing with Virtual Peers: Bootstrapping Contingent Discourse in Children with Autism” Proc. Eighth Int'l Conf. Learning Sciences, vol. 2, pp. 382-389, 2008.
[150] R. Feldman, “Coding Interactive Behavior Manual (CIB),” unpublished manuscript, Ramat-Gan, Israel, Bar Ilan Univ., 1998.
[151] M.J. Richardson, K.L. Marsh, and R. Schmidt, “Effects of Visual and Verbal Interaction on Unintentional Interpersonal Coordination,” J. Experimental Psychology: Human Perception and Performance, vol. 31, no. 1, pp. 62-79, 2005.