Shrikanth “Shri” Narayanan is a luminary of affective computing at the University of Southern California (USC) as the University Professor and the Niki & C. L. Max Nikias Chair in Engineering. He holds various appointments such as Professor of Electrical & Computer Engineering, Computer Science, Linguistics, Psychology, Neuroscience, Pediatrics, and Otolaryngology—Head & Neck Surgery and Research Director of the Information Sciences Institute. His journey, which first started as an internship at AT&T Bell Labs, displays a dedicated commitment to understanding the human condition through the power of engineering and computing technology. He is an IEEE Fellow, has published over 1000 papers, and his inventions/patents have led to technology commercialization.
In honor of his many achievements, he has received the 2024 Edward J. McCluskey Technical Achievement Award for, “…pioneering contributions to speech, language and multimedia processing and affective computing and their human-centered societal applications.“
Congratulations on recently receiving the Edward J. McCluskey Technical Achievement Award! How do recognitions such as this inspire and motivate you in your ongoing research endeavors?
While honors such as these are exciting, they are so humbling because of the realization of two things: first, these are recognitions that represent the collective efforts of numerous minds, and second, be just fortunate and grateful to be selected from among the incredible colleagues in the field. I have had the privileged opportunity to explore many interdisciplinary questions through an engineering and computing lens, primarily centered on humans, and often getting outside the proverbial comfort zone. Being recognized for such research, and more importantly, for novel topics and approaches, indeed inspires us to forge forward on themes that may seemingly be non-mainstream, long haul, and risky — for example, topics on human emotions and mental health. I hope this may signal encouragement to continue such a journey not just to me, but others who would like to pursue similar topics, especially early in their careers.
Honor your colleagues’ achievements. Nominate Someone for a Major Award Today!
With over 1000 papers published, is there a specific work or pivotal moment in your career that stands out to you the most or are most proud of?
This is a difficult question! I have been lucky with a series of key moments in my career and research so far – starting with my first internship at the incredible AT&T Bell Labs in Murray Hill that opened up so many intellectual pathways for me, to the exciting moment of getting the speed of magnetic resonance imaging to enable capturing the amazingly choreographed action of the vocal instrument during speech. This led to new scientific insights and technology and clinical applications. The other key moments include work focused on computationally illuminating how someone is communicating expressively with effect beyond merely what they are communicating, work on building speech and conversational technologies for children across wide-ranging applications from healthcare to learning to creating technologies in supporting research and clinical care in mental health. None of these would have been possible without the amazing collaborators.
Your research encompasses engineering and computer science, linguistics, psychology, and beyond! How has this interdisciplinary approach contributed to the development of human-centered technologies?
The core of my work is focused on understanding the human condition through engineering approaches, and in developing technologies that can support and enhance human experiences. This includes understanding aspects of cognition, including how humans produce, process, and interact using language, human emotional experience/expression/perception, and broader aspects of human behavior, notably when affected by disease and disorder. Advances in this endeavor have been based on a strong grounding in the vast scholarship and advances in fields such as linguistics, psychology, behavioral and biomedical sciences to frame and interpret the research, as well as in turn advance basic theory in these fields.
Can you share some specific examples of how these technologies have been applied to support and enhance the human experience in areas such as defense/intelligence, health, education, and the media arts?
One of the early examples in support of national security/defense needs is the development of speech-to-speech translation systems that would mediate communication between individuals who do not share the same language, such as during a medical mission in the field. Another example is one where we have also developed several computational methods that quantify diagnostic measures that complement traditional questionnaires or interviews for a variety of mental and behavioral health domains. For example, we have contributed AI methods to understand the details of psychotherapy mechanisms, and their relation to outcomes, leading to novel ways of ensuring quality assurance, including supporting the training of clinical counselors and therapists. In education, we have developed methods for automatically assessing the reading abilities of children from diverse linguistic backgrounds. In the arts, we have developed methods that have illuminated vocal song production in a range of genres from opera to beatboxing, as well as characterized how music is experienced by individuals at the brain-body-behavior levels. We have also developed methods to understand the representation and portrayals of characters in stories we tell in entertainment media –film and television—using audio, video, and language processing and machine learning.
How do you envision the continued impact of your work in these domains (mentioned above) and what challenges or opportunities do you foresee for the future?
Advances in computing will continue to have a tremendous societal impact globally, including in critical human realms such as health and well-being. Enabling and realizing this possibility, however, requires continuing engagement between all stakeholders right from the get-go: to both incorporate domain knowledge (e.g., from medicine, and education into computational approaches) and in turn advance science empowered by computation.
In the development of AI-based technologies, ethical considerations are crucial. How do you approach ethical considerations in your work, especially in fields like affective computing?
I believe ethical considerations should be a central part of the research process – right from defining and framing the research questions to making the design and methodological choices to their implementation and evaluation. Critically, this would benefit from incorporating diverse perspectives, in a participatory way, including the potential users of the technology. Also, it is important to underscore the need to have an open mind to continually evaluate the impact of the technology, especially the unexpected and unintended ones, as research advances are made.
Your extensive career spans academia, industry, and research institutions. Can you share insights into the connections and partnerships you’ve developed throughout your career, and how these connections have influenced your research or career path?
In our field of human-focused research and innovation, directly centered on computing and technological advances that aim to benefit society, collaborations and partnerships are integral. In my work, close partnerships with industry have helped to propel forward-looking research with the cutting-edge technology afforded by industry partners. Additionally, these have helped provide the students with valuable experiences that have enriched and elevated their research and offered them professional growth opportunities. Other community partners have been crucial in offering a platform for defining and disseminating research findings that can have a direct societal connection and impact, such as healthcare providers, storytellers, and K-12 educators. I have benefitted tremendously from such partnerships from extensive engagement with industry, including starting as a student intern at AT&T to a visiting faculty researcher at Google Research presently.
More About Shrikanth Narayanan
Shrikanth (Shri) Narayanan is USC University Professor and Niki & C. L. Max Niki as Chair in Engineering at the University of Southern California (USC), where he holds appointments as Professor of Electrical & Computer Engineering, Computer Science, Linguistics, Psychology, Neuroscience, Pediatrics, and Otolaryngology—Head & Neck Surgery and Research Director of the Information Sciences Institute. Before USC, he was with AT&T Bell Labs and AT&T Research. His interdisciplinary research focuses on developing engineering and computing approaches to understand the human condition and in creating human-centered technologies to support and enhance human experiences, across applications with direct societal relevance including in defense/intelligence, health, education, and the media arts. He is a Fellow of the Acoustical Society of America, IEEE, the International Speech Communication Association (ISCA), the American Association for the Advancement of Science, the Association for Psychological Science, the Association for the Advancement of Affective Computing, the American Institute for Medical and Biological Engineering, and the National Academy of Inventors. He is a Guggenheim Fellow and member of the European Academy of Sciences and Arts, and a recipient of many awards for research and education including the 2023 ISCA Medal for Scientific Achievement and the 2023 Richard Deswarte Prize in Digital History. He has published widely (over 1000 papers) and his inventions/patents have led to technology commercialization including through startups he co-founded: Behavioral Signals Technologies focused on AI-based conversational assistance and Lyssn focused on mental health care and quality assurance.
