Kaoru Sumi

Prof. Kaoru Sumi is a professor and interdisciplinary researcher specializing in human-agent interaction, affective computing, persuasive technology, and intelligent systems. With a Ph.D. in Engineering from the University of Tokyo and over 25 years of academic experience, she designs emotionally engaging, socially impactful technologies that bridge people and machines across both physical and virtual environments.

Her research explores how humans interact with intelligent agents and how those interactions are shaped by emotion, cognition, and context. She extends this work into immersive environments, examining how metaverse experiences influence communication, learning, and sensory perception. Her recent research focuses on phantom sense phenomena, investigating the conditions under which people feel virtual touch or presence and the implications for embodied interaction.

Prof. Sumi’s work spans HCI, AI, XR technologies, and emotion-sensitive interfaces, contributing to education, digital storytelling, healthcare, and behavior design. She has published over 60 peer-reviewed journal articles and 80 international conference papers, earning multiple Best Paper Awards.

She has led major research projects funded by MEXT, NEDO, and other agencies. She is currently General Chair of Persuasive Technology 2026, Associate Editor of Frontiers in Artificial Intelligence and Behaviour & Information Technology, and a Senior Member of IEEE and ACM.

She also contributes to IEEE’s diversity efforts as WIE Coordinator for IEEE Japan Council and Chair of the IEEE Sapporo WIE Section.

Prof. Sumi has supervised over 50 graduate students and teaches interdisciplinary courses in English and Japanese. Her teaching emphasizes project-based learning, empathy-driven design, and the use of avatars, virtual agents, and embodied experiences.

A frequent keynote speaker, she shares insights globally on emotional interaction with intelligent systems, phantom sensation in virtual spaces, and the future of metaverse-based experiences.

Contact: kaoru.sumi@ieee.org

Website: https://www.fun.ac.jp/en/faculty/sumi-kaoru

Abstracts

Phantom Sensations and Social Cues in the Metaverse: Rethinking Embodied Learning and Agent Design

Abstract: How do emerging technologies in the metaverse reshape the way we feel, learn, and engage with others? This talk presents findings from recent experiments examining phantom sensations and social agent-based learning within immersive virtual environments.

In one study, participants in a VR-based metaverse platform experienced phantom sensations—such as warmth, falling, or collision—without any physical stimuli. These pseudo-haptic responses, triggered by visual and narrative cues, increased user immersion, emotional engagement, and the motivation to revisit virtual spaces. Although these effects vary by individual, the findings suggest that designers of educational and experiential systems in the metaverse should carefully consider the cognitive and emotional consequences of such sensations.

Complementing this work, we also examine how the appearance and behavior of avatars—representing diligent or disengaged learners—affect other users in shared virtual classrooms. Even without direct interaction, avatar presence shaped learners’ motivation and engagement, particularly among those sensitive to social cues or peer modeling. These insights highlight the power of agent design in influencing learning outcomes in subtle but meaningful ways.

Drawing from these studies, the talk reflects on key considerations for the future of metaverse-based learning: What kinds of sensory and social experiences support deep, inclusive, and ethical learning? How should we design agents and environments that not only inform but emotionally resonate? By integrating phantom sensation theory, social cognition, and affective computing, we propose a new framework for embodied education in the metaverse.

Emotionally Expressive Pet-Like Artifacts for Behavior Change: Non-Verbal Persuasion through Empathic Design

Abstract: As persuasive technology increasingly moves beyond humanoid agents, emotionally expressive, pet-like artifacts have emerged as an intuitive and low-pressure medium for influencing human behavior. This talk explores the design and evaluation of a dog-shaped virtual agent that encourages behavior change through non-verbal emotional expressions. Drawing on canine-inspired communication, the artifact expresses emotions such as sadness, confusion, and joy using only posture, movement, and vocalization.

Using a Mixed Reality (MR) environment, we conducted a study in which participants interacted with the virtual pet and were subtly encouraged to perform a simple task—disposing of trash—without receiving any verbal instruction. Results showed that emotions like sadness and confusion were highly effective in evoking empathy and prompting voluntary action, while anger and joy were less persuasive due to emotional ambiguity or avoidance responses.

The talk will discuss how emotion clarity, rather than animation complexity, plays a critical role in non-verbal persuasion. We introduce the concept of Affective Learning, in which emotionally expressive agents serve as learning companions that encourage motivation and attention through empathetic cues. These findings offer new directions for designing emotionally intelligent, versatile systems for education, healthcare, and sustainable behavior change.