The Community for Technology Leaders

Affective Computing: From Laughter to IEEE

Rosalind W. Picard, IEEE

Pages: pp. 11-17

Abstract—This is an invited introduction to the first issue of the IEEE Transactions on Affective Computing, telling personal stories and sharing viewpoints of a pioneer and visionary of the field of Affective Computing. This article is not intended to be a thorough or a historical account of the development of the field, for the author is not a historian and cannot begin to properly credit the extraordinary efforts of hundreds of people who helped to cultivate and expand the rich and fertile landscape that extends before us now.

Index Terms—Affective Computing, agents, autism, psychophysiology, wearable computing.


Jodie is a young woman I am talking with at a fascinating annual retreat organized by autistic people for autistic people and their friends. Like most people on the autism spectrum (and many neurotypicals, a term for people who don't have a diagnosed developmental disorder), she struggles with stress when unpredictable things happen. Tonight we are looking at what happened to her emotional arousal as measured by a wristband that gathers three signals—skin conductance, motion, and temperature ( Fig. 1). Jodie says she was upset to learn that the event she was supposed to speak at was delayed from 8:00 to 8:30 pm. She started pacing until her friend told her that was not helping and to stop. Many people don't have an accurate read on what they are feeling (this is known as alexithymia) and while she thought pacing helped, she wasn't certain, so she took his advice. She then started to make repetitive movements often seen in autism, commonly called “stimming,” and continued these until the event began at 8:30. In Fig. 1, we see her skin conductance on the top graph, going down when she was pacing, up when she was stimming, and hitting its highest peaks while she gives her presentation. The level also stays high afterward during other people's presentations, when she stayed up front to handle problems everyone was having with the audio-visual technology.


Figure    Fig. 1. Skin conductance level (top graph). Skin surface temperature (middle graph) and 3-Axis accelerometer values (lower graph). Skin conductance, which is associated with emotional arousal, was lowered during pacing, while it went up during “stimming,” a presentation, and (afterward) while dealing with some audio-visual equipment problems. This data is from a young adult on the autism spectrum.

Collecting data related to emotional arousal is not new: For example, skin conductance has been studied for more than a hundred years. What is new, however, is how technology can now measure, communicate, adapt to, be adapted by, and transform emotion. Powerful new things can be done with these abilities. For example, Jodie collected her emotional arousal data wearing a stretchy wristband, clicked to upload it into a mobile viewer, let her friend (who asked her to stop pacing) see the data, and the first words out of his mouth were, “I'm not going to tell you to stop pacing anymore.” The next morning I saw her pacing without his interference. The ability to communicate objective data related to her emotional arousal and activity—specifically her sympathetic nervous system activation, of which skin conductance is a sensitive measure, prompted a change in his behavior. Mind you, she had told him in the moment of stress that she thought pacing was helping, but this did not change his behavior. Information about emotions that is objective carries much more power than self-reported subjective feelings.

The convenience of a new affective computing technology can lead to new self-understanding, to improved communication between people, and to much more, including (if researchers make it so) to new technologies that reduce stress instead of increasing it. There's a saying “if you can't measure it you can't manage it.” Measuring the frustration caused by a technology when it happens can enable engineers to pinpoint what causes it and work to prevent or reduce it. Technology can also be improved if it has an intelligent ability to respond to emotion, and technology can be improved by virtue of incorporating principles of emotion learned from biological systems. But there are many extraordinarily hard challenges to solve in order to bring about new benefits.

Attitudes toward affective computing, which I defined in 1995 as “computing that relates to, arises from, and deliberately influences emotion,” have changed so much in the last decade that it is now hard for some people to believe it used to be a ludicrous idea. In the 1990s, I had never heard of the shorthand “LOL” (Laugh out Loud), but it applied to this research. I beg the reader to let me indulge in some remembrances, starting in 1991, my first year on the MIT faculty.

Inthe Beginning, Laughter...

One morning over breakfast cereal and the Wall Street Journal (the only nontechnical journal I regularly read) a front-page article about Manfred Clynes caught my eye. He was described as a brilliant inventor who, among better-known inventions that became commercially and scientifically successful, also invented a machine for measuring emotion. His “sentograph” ( sentire is Latin for “to feel”) measured slight changes in directional pressure applied to an immovable button that a person pushed on. The finger pushes showed characteristic patterns related to joy, sadness, anger, sex, reverence, and more. This is not a list approved by mainstream emotion theorists—they don't include sex or reverence—and Manfred is far from mainstream, a child prodigy with a fan letter from Einstein for his piano playing and coauthor on the 1960 paper that coined the word “cyborg.” But he measured emotion, and later, the measures were replicated by others. I was amused by this crazy fact, although not enough to do anything more than file the article. The article mentioned my friend, Marvin Minsky, who many years later introduced me to Manfred, and we then became friends.

Manfred never claimed to be the first to build a machine to measure emotional categories—nobody knows who did it first—but he did tell me that when he first tried to present his ideas about measuring emotion on stage, to other scientists, he was laughed at by the audience, and it was not the kind of laughter most speakers crave. He said he was literally laughed off the stage.

2.1 Discovering Real Importance for Emotion

When I first started thinking about emotion it was the last thing I wanted to think about. I was up for tenure at MIT, working hard raising money and conducting what people later wrote was pioneering research in image and video pattern modeling. I was busy working six days and nights a week building the world's first content-based retrieval system, creating and mixing mathematical models from image compression, computer vision, texture modeling, statistical physics, machine learning, and ideas from film-making, and spending all my spare cycles advising students, building and teaching new classes, publishing, reading, reviewing, and serving on non-stop conference and lab committees. I worked hard to be taken as the serious researcher I was, and I had raised over a million dollars in funding for my group's work. The last thing I wanted was to wreck it all and be associated with emotion. Heck, I was a woman coming from engineering. I did not want to be associated with “emotional,” which I took at the time to mean irrational.

However, I kept running into engineering problems that needed...well, something I did not want them to need. For example, working on computer vision I knew that we had a lot to learn from human vision, and I collaborated with human vision scientists who focused on the cortex and visual perception. We labored to build computer vision systems that could see like people see, and learned to build banks of filters, for example, that could detect high-contrast oriented regions and motions in ways that seemed to be similar to stages of the human visual cortex. Much engineering, whether for vision, or, earlier in my life, for computer architectures, was focused on trying to replicate the amazing human cortex. We all wanted to figure it out by building it. But nowhere did any of these methods address a problem we were running into: How do you find what is interesting for a person? How do you find what matters to them? How do visual attention systems figure this out and shift automatically when they need to shift? Building a vision system is not just about detecting high-contrast oriented lines or telling a dog from a cat.

Another problem arose from my years of work at AT&T Bell Labs and at MIT building new kinds of computer architectures for digital signal processing. We came up with many clever ways to parallelize, pipeline, optimize, and otherwise process sounds and sights and other signals humans usually interpret effortlessly. However, never did anyone figure out how to give a computer anything like motivation, drive, and a sense of how to evaluate shifting priorities in a way that acted genuinely intelligent or that genuinely cared about anything. We could give it functional programs that approximated some things like this—under limited conditions, following a brittle mathematical specification that covered all the cases known up front—but this always failed pathetically when encountering something new. And it didn't scale—the space of possibilities it needed to consider became intractable. Today we know that biological emotion systems help human beings handle complex unpredictable inputs in real time. Today we know that emotions signal what matters, what you care about. Today we know emotion is involved in rational decision-making and action selection, and in order to behave rationally in real life you need to have a properly functioning emotion system. But at that time, this was not even on the radar. Emotion was irrational and, if you were smart, you didn't want to have anything to do with it.

Most surprising to me was when I learned that emotion interacts deeply in the brain with perception—a process that I thought, at least from my dealings with human vision researchers, was driven by the cortex. But one Christmas, while reading Richard Cytowic's “The Man Who Tasted Shapes,” I was jolted out of my cortex-centric beliefs by his findings that, at least in synesthesia (when senses appear to cross and a person feels shapes in his palms when tasting soup, or sees colors with letters involuntarily or other crossed modalities), the cortex was in some cases showing less activity, not more. Cytowic argued that multimodal perception was also happening in the limbic structures of the brain, regions physically below the cortex, which were known to be important for three things: attention, memory, and emotion. I was interested in attention and memory, so I got my hands on more neuroscience literature about these limbic regions, only to find that the third role—emotion— kept coming up as essential. Emotion played major roles not only in perception, but also in memory and attention, in rational deicison making, and in human-machine interaction. Emotions influence action selection, language, and whether or not you decide to double-check your mathematical derivations or comment your computer code.

Emotion being useful was not what I was looking for. I became uneasy. I did not want to work on or be associated with emotion, yet emotion was starting to look vital for solving the hard engineering problems we needed to solve. A scientist has to find what is true, not just do what is popular. I was becoming quietly convinced that engineering dreams to build intelligent machines would never succeed without incorporating insights about emotion. I knew somebody had to educate people about the evidence I was collecting and act on it. But I did not want to risk my reputation and I was too busy. I started looking around, trying to find somebody, ideally male and established, whom I could convince to develop this topic, which clearly needed more attention than I had time for.

2.2 Who Wants to Risk Ruining Their Reputation?

I screwed up my courage and invited Jerry Wiesner, former president of MIT and Scientific Advisor to Presidents Eisenhower, Kennedy, and Johnson, to lunch. Jerry was in a suit and always seemed very serious and authoritative. Over fish and bonbons at Legal Sea Foods I asked him what was the most important advice he had for junior faculty at MIT. I strained to hear him over the noise of that always too-loud restaurant, but one line came out clear: “You should take risks. This is the time to take risks.” As I walked back the one block to the lab, I took a detour and did some thinking about this. I was working in an exciting new research area at the time—content-based retrieval—but I didn't think it was really risky.

The Media Lab saw me as one of their more conventional players, as “the electrical engineer.” Nicholas Negroponte, architect and founding director, spoke with pride and perfect French pronounciation of how he formed the Media Lab as a “Sah-lon de ref-oos-say.” The orginal Salon des Refusés was an exhibition by artists of work that was rejected by the authorities in charge. Nicholas was proud of establishing a lab that would do research that others might laugh at and reject. I wasn't quite sure I wanted to be labeled as a rejected misfit, but I didn't learn he saw our faculty in this way until after I was already a member of the lab. It was freeing to hear that if I was indeed viewed as a misfit, it would be valued. If I chose to work on emotion, the misfit title was going to happen. Maybe it would be okay here.

One of the brilliant visionaries Nicholas had recruited to the Media Lab was Seymour Papert, mathematician and leading thinker in education and technology, who told our faculty about the researchers in days long ago who were all focused on trying to build a better wagon, making the wheels stronger so they stayed round and so they didn't break or fall off as easily, or otherwise working hard to make wagons last longer, go faster, give smoother rides, and cover more distance. Meanwhile, Seymour said that while all the researchers of that day were improving the wagon and human-wagon interaction, those crazy Wright brothers went off and invented the airplane. He said we faculty in the Media Lab should be the crazies inventing the new way to fly. My maiden name is Wright and this story was inspiring and reassuring.

Convinced that emotion was really important and people should start paying attention to it, and that maybe my lab wouldn't mind if I detoured a few weeks to address this topic, I spent the holidays and some of the January “Independent Activities Period” and wrote a thought piece that I entitled “Affective Computing” to collect my arguments, and circulated it as a tech note quietly among some open minds in the lab. A student from another group read it and showed up at my door with a stack of six books on emotion, “You should read these,” he said. I did. I love how the students at MIT tell the faculty what to do. I needed to hear what he said.

I then read every book on emotion I could get from Harvard, MIT, and the local library network only to learn that psychologists had more than a hundred definitions of emotion, nobody agreed on what emotion was, and almost everyone relied on questionnaires to measure emotion, despite knowing the self-reports collected were unreliable and sometimes totally inaccurate. I went to Jerry Kagan in the Psychology Department at Harvard to talk about more accurate and systematic ways to measure and characterize affective information. He gave me a hard time at first but in the end he was very nice and almost encouraging: He told me “You're shooting for the moon” when I proposed that my team could build wearable technology to measure and characterize aspects of emotion as it naturally occurred in daily life. I thought psychologists could benefit from the systematic approach engineers typically bring to hard problems.

I attended neuroscience talks and read key findings on emotion in the neuroscience literature and found their methods to be more concrete—showing evidence for precise pathways by which aspects of emotional perception and learning appeared to be happening, and this was compelling, especially findings like Joe LeDoux's that showed perceptual learning (e.g., a rat learning to fear a tone) without involving the usual cortical components (e.g., after the audio cortex had been removed). Antonio Damasio's book Descarte's Error was also inspirational, arguing for the role of emotion in rational decision making and behavior.

I spruced up my tech note envisioning Affective Computing as a broad area that I thought engineers, computer scientists, and many others should consider working on, and submitted it as a manifesto to a non-IEEE journal that had traditionally printed bold new ideas. It was rejected and one of the reviews indicated that the content was better suited to an in-flight magazine. I could hear laughter between their lines. I gave a talk on the ideas to our computer vision research group and people were unusually silent. This was what I feared.

I gave a copy of the thought piece to Andy Lippman, a tall energetic man who always has bountiful words for sharing his opinions. Usually we talked about signal processing or video processing. One day he showed up in my doorway, silent, with a peculiar look on his face, holding a document. He stabbed it with his index finger, shook his head, pointed at it, shook his head more, but said nothing. This was not like him. “Is something the matter?” I angled my head, wondering if he'd lost his voice. He blurted, ”This is crazy! CRAZY!” He looked upset, and I couldn't see what he was pointing at. I hesitated, “Uh, crazy is, good, in the Media Lab, right?” He nodded like a Bostonian being asked if he'd like free ice cream with mix-ins. Then I saw what the document was: It was my affective computing paper. He waved it, shook his head again, and left with an odd smile. I never did resubmit or get that first tech report published, but I did succeed in tongue-tying the voluble Lippman.

2.3 Visionary Supporters Trump Peer Review

I am a big fan of peer review, and work hard to maintain the integrity of that process. But there are times in the life of new ideas when peer-reviewed papers don't stand a chance of getting published. Sometimes years of acclimation are needed before an idea can make it through the process, even if the work is done solidly and with the best science and engineering. I realized the early ideas on affective computing were not going to make it into print until a lot more work had been done to prove them, and I only had a year before I was up for tenure. How could I get a whole set of new ideas out when the average time from submission to publication of my computer vision papers was measured in years? Nicholas Negroponte invited me to coauthor his Wired column on affective computing. We published it and got a mix of responses—the most memorable being letters from people declaring, “You are at MIT, you can't know anything about emotion.” Wired was no substitute for peer review, but it started to get my ideas out.

David Stork invited me to author the chapter on Hal's emotions for the book Hal's Legacy, commemorating the famous computer in Stanley Kubrick and Arthur C. Clarke's film 2001 A Space Odyssey. All of the other chapters addressed attributes of Hal like his chess playing ability, his speech, his vision, etc., and had “the most famous person in the field” to write them. David and I joked that I was the only person at the time that he thought of as representing the field of computers and emotions, and the word “field” was used with a friendly grin. I still enjoyed being in the book with a lot of impressive colleagues—Ray Kurzweil, Don Norman, Daniel Dennett, and others, and it was encouraging to be grouped with so many successful scientists. However, when I had dinner with Ray Kurzweil, his wife asked me if I was the “emotion woman,” which confirmed more of my worries. But I had started digging deeper into affective computing research and I knew the work was needed, even if it wrecked my image.

The famous scientist Peter Hart, after coaxing me to ride with him up the “hill” (it felt more like a mountain) of Old La Honda on a 105 degree July day, told me he thought Affective Computing was going to become very important, and he encouraged me to drop all the research I'd just raised over a million dollars in funding for (content-based retrieval) and pursue affective computing wholeheartedly. I feverishly wondered how I could ever do that. He hosted, in July 1995, at Ricoh Silicon Valley, what was the first presentation outside of MIT on the ideas that would become my book Affective Computing. I saw Peter as an established outside authority in Pattern Recognition, not just a Media Lab crazy type, and his encouragement enabled me to believe that a book and more serious dedicated work in this field might be worthwhile. At least he would be one respected technical researcher who wouldn't write me off.

In August 1995, I e-mailed the director of the Media Lab that I was changing the name of my research group at MIT to “Affective Computing.” He said it was a very nice name, “gets you thinking,” and “is nicely confused with effective.” I liked the thought that my crazy new work would be confused with being effective.

I was asked to fax my unpublished tech report to Arthur C. Clarke (he didn't do e-mail!). I faxed it and he mailed me a personal letter saying he liked it and adding, “I sent your paper to Stanley—he is working on a movie about AI.” I never got to meet “Stanley,” but I understand he was the brilliant mind behind giving HAL emotions in the film 2001. When I read the original screenplay, it had almost nothing on emotion in it, and Clarke's subsequent book on the story also downplayed emotion. But in the film, HAL showed more emotion than any of the human actors. Through my Media Lab connections like Clarke, I started to see that there were many mavericks who had recognized the power and importance of emotion, even though there were many more who couldn't understand why it mattered. I felt encouraged to push ahead in this area, despite the fact that I heard my technical colleagues at conferences whispering behind my back, “Did you hear what weird stuff she's working on?” and some of them blushed when I looked up at them and they realized I'd overheard. (I did feel vindicated five years later when one of them asked me if I would share my affect data with him as he was starting to do work on affective analysis.)

Public Broadcasting TV producer Graham Chedd for Scientific American Frontiers came by with one of my favorite actors, Alan Alda, and got interested in what my team was doing and included our very early affective research in two of their shows. I am told that these episodes still air, on very late night television, where you can see Alan Alda's emotional arousal going up as he thinks about hot red peppers and going down while he thinks about Saltine crackers, while I'm standing next to him, pregnant with my first child. Somehow it now seems fitting for late night television.

Dan Goleman called from the New York Times during a very busy week and I asked him if we could talk at a different time. He said he was going to write about our work that week whether I would make time to speak with him or not. Later his book on Emotional Intelligence sold over 5 million copies. Putting “emotional” and “intelligence” together was a brilliant combination, and while it sounds normal today, at the time it was like an oxymoron. His writing did a lot to interest the general public in the important roles emotions play in many areas of success in life—he argued it was more important than verbal and mathematical intelligences, which of course was what AI researchers had been focused on. The topic of emotion was starting to get more respect, although it was still very hard to get computer scientists to take it seriously.

Much later, William Shatner came by my office, dragged by his ghost writer to write about the science of Star Trek and the role of emotion in their shows. It was kind of a stretch to find some science there, given the booming sounds in the vacuum of outer space, and more, but I did confirm that the character of Spock had emotion, which (despite his not being expressive and keeping it under control) was important for his intelligent functioning to be scientifically accurate. If he really didn't have emotion and behaved as intelligently as he behaved, then it would have been bad science in the show. Leonard Nimoy later came to MIT and hosted a big event I chaired featuring new technology measuring and communicating emotional signals. He appeared remarkably unemotional, even when he was not playing Spock. His presence attracted more people to come and learn about why my group was developing affective technologies.

A famous high-price-charging speaker's bureau invited me to join their list of speakers, offering lots of money if I would give talks about “more broadly interesting” technology topics than affect and computing. They thought emotion was not going to be of sufficiently broad interest to their well-heeled clients. I knew at this point I was going to spend all my spare cycles trying to get high quality research done on affective computing, and trying to get more engineers and computer scientists to consider working on emotion, so I declined their offer. I started giving more talks than ever—dozens every year, mostly with zero or low pay to academic groups to try to interest them in working on affective computing.

I remember one talk where Larry Rabiner came up to me afterward and asked why I was working on emotion—he said, “it's a very hard problem to tackle, and it just doesn't matter—why are you wasting time on it?” I don't think he had paid much attention in my talk, or I had done a very bad job of explaining. I had always admired Larry's work and this was tough to hear, but I tried to explain why I thought it was critical in early development for learning of language. I pointed out dogs and small infants seem to respond to affect in speech. He did listen, but I never heard from him again.

After another talk, I remember a world-famous MIT computer scientist coming up to me, agitated, looking at my feet the whole time and complaining to me, “Why are you working on emotion? It's irrelevant!” (I'm told this is how you tell if a CS professor is extroverted or introverted—if he looks at his feet, he's introverted, if he looks at yours, he's extroverted.) I wasn't able to convince him of its value, and he was soon joined by others who looked at each other's feet and changed the subject to help calm him down. On multiple occasions, colleagues confided in me that they didn't know what emotion really was (other than extreme emotions like anger) and some of them even said, “I don't have feelings and I don't believe they have a physical component you can measure.” I think one of the attractions of computer science to many of them was that it was a world of logic largely devoid of emotional requirements, and they didn't want this threatened.

Through my talks to various groups, I became increasingly convinced that affective computing needed to be addressed, even if most computer scientists thought emotion was irrelevant. I wanted to make affective computing interesting and respectable so that progress would be made. I was always encouraged when people would go from looking scared of the topic, as if it was going to be an embarrassing talk to be seen at, to wanting to spend lots of time with me afterward talking deeply about the subject.

Somehow in the midst of all of this, while up for tenure, trying to build and move into a new house, and getting ready to give birth to my first son, I signed a book contract in 1996, moved into the house, delivered the baby, delivered the book nine months later, and submitted my tenure case to MIT with a freshly minted copy of Affective Computing. At the time I had no peer-reviewed journal papers related to affective computing; those would come later. All my peer-reviewed scientific articles were on mathematical models for content-based retrieval or were conference papers on affective signal analysis. I was told that reviewers didn't know what to make of my schizophrenic tenure case—they wondered if the book was authored by somebody different than the person who wrote the papers, as if Rosalind Picard was a common name and maybe there was a mistake. But the Media Lab loved it, and was probably the only place on the planet that would have felt that way. The director of our lab phoned me and said “your tenure case went through like a hot knife through butter.” The risk I had taken to start out in an almost totally new area a year before submitting my tenure case had certainly not hurt my career. But I never did it for my career, I did it because I believed then, and I still believe, that affective computing is an extremely important area of research.

I was also amazed how, over time, the appeal of the topic became very broad—not just to researchers in computer science and human computer interaction, but also in medicine, literature, psychology, philosophy, marketing, and more. I had never known there were so many communities interested in affect and I started to engage with researchers in a huge number of fields. I have learned a ton doing this, and it has been mind expanding.

I was delighted to see workshops on Affective Computing springing up around the world, led by visionary colleagues in computer science and psychology. I did not help much in terms of organizing meetings and admire greatly the huge efforts put in by so many talented technical colleagues who truly fostered the growth of this field. I cannot properly name them all here; however, Klaus Scherer, Paolo Petta, Robert Trappl, Lola Canamero, Eva Hudlicka, Jean-Marc Fellous, Christine Lisetti, Fiorella de Rosis, Ana Paiva, Jianhua Tao, Juan Velasquez, and Tienu Tan played especially important and memorable roles instigating some of the early scientific gatherings. Aaron Sloman, Andrew Ortony, and I were frequent speakers at these gatherings, and I enjoyed their philosophical and cognitive perspectives.

The HUMAINE initiative became very influential in funding significant European research on emotion and computing, propelling them ahead of research efforts in the United States. The community involved a lot of top researchers under the warm leadership of Roddie Cowie, and, with the expert technical support of Marc Schroeder, was well organized and productive, funding dozens of groundbreaking projects. The US did not seem as willing as Europe to take bold risks in this new research area and I always wondered why we lagged so far behind Europe in recognizing the importance of affect. I was lucky to have Media Lab corporate consortium funding with “no strings attached” or our MIT Affective Computing group would never have been able to get up and running. Meanwhile, a US National Cancer Institute grant supported Stacy Marsella at USC in developing a pedagogical system to teach emotion coping strategies to mothers of pediatric cancer patients and an US Army Research Institute grant recognized the importance of putting emotions into the cognitive architecture Soar (work by Paul Rosenbloom, also at USC, which not only included Jonathan Gratch, but also hooked him on emotion). Much later the US National Science Foudation funded work by Art Graesser at Memphis that included my lab helping develop emotion recognition tools for an intelligent tutor, and then still later, work by Rana el Kaliouby and Matthew Goodwin and me building affective technology for autism. While I remain very grateful for all sources of funding, I especially am grateful for sources that give scientists the freedom to try things before the ordinary peer-review and proposal-review processes are ready to accept them. IEEE and Beyond

I have a long history with the IEEE, from joining as a student to decades later being honored as a fellow. I played a small role in helping found the wearable computing conference ISWC and the wearables special interest group, served on dozens of program committees, organized workshops, and served as guest editor and associate editor of the IEEE Transactions on Pattern Analysis and Machine Intelligence. I've reviewed stacks of IEEE papers so high they could bury a poor innocent bystander if they toppled. I know the IEEE research community. However, when I submitted my first emotion recognition paper focusing on physiological pattern analysis to the IEEE conference on “computer vision and pattern analysis” the reviewers wrote “the topic does not fit into CVPR since it doesn't have any computer vision in it.” Later I strategically put “Digital processing of...” and “Signal processing for...” in the titles of papers submitted to the IEEE International Conference on Acoustics, Speech, and Signal Processing and they got accepted. This same trick worked to get past the “it doesn't fit” complaints for our first IEEE Transactions on Pattern Analysis and Machine Intelligence paper on affective computing as well: I put “machine intelligence” in the title. Of course, it was not that easy: The editor also insisted that five thorough reviewers iterate with me before approving the paper (usually three will suffice). I had been an associate editor of TPAMI and seen a lot of reviews, but never any set of such length. I addressed every comment and the paper got published.

By the way, it was not just the IEEE—the ACM also rejected my first affective computing submission as “not matching any of the topics or themes in the human-computer area.” I wondered from the review if they had even read the paper or just rejected it when they saw “emotion” in it. Years later I was delighted when several affective topics were added to their official themes. To this day, I still feel slightly amazed when I see conferences that openly solicit affective topics, even though Affective Computing has its own international conference now and many other conferences also openly solicit affective com-puting work. It just wasn't always that way—in the beginning, emotion was really fringe, and the few people working on it had to have an unusually large allocation of self-confidence. And, now here is the first issue of the IEEE Transactions on Affective Computing, which truly presents the field as respectable, provided that participants continue to uphold the IEEE's highest standards. Wow.

Whether or not Affective Computing is an area in which you conduct research, you are using emotion when you choose where to spend your time—when you choose to act on what matters most to you. Affective computing researchers have a chance to elucidate how emotion works: how to build it, how to measure it, how to help people better communicate and understand it, how to use this knowledge to engineer smarter technology, and how to use it to create experiences that improve lives. Affective computing is a powerful and deeply important topic, full of extremely difficult technical, scientific, philosophical, and ethical challenges. I believe it contains the most complex real-time problems to be solved in computer science, and at the same time it is much more than a subset of computer science. The complexity and challenge of giving computers real-time skills for understanding and responding intelli-gently to complex naturally-occuring and naturally-expressed human emotion spans many fields, including the human sciences of neuroscience, physiology, and psychology, and is not a topic to be treated lightly, although laughter remains one of my favorite emotional expressions.

Congratulations, Jon Gratch and the talented IEEE Transactions on Affective Computing Editoral Board. I feel honored to be asked to write my personal remembrances for this first issue, and I wish you all much success with this groundbreaking new journal.


The author wishes to thank all her graduate and undergraduate student researchers over the years, especially those who helped build a solid base of research in Affective Computing, and those who politely tolerated and supported the group's transition to this topic back when they thought emotion was embarrassing and wished their advisor would go back to doing normal work. She also cannot begin to properly credit the remarkable learning environment that MIT and the Media Lab have created, supporting people who have different ideas, even laughable ones. MIT and the Media Lab are truy special places full of amazing colleagues. The author would also like to thank Drs. Ted Selker, Cynthia Breazeal, Rich Fletcher, Rana el Ka-liouby, and Matthew Goodwin for their significant collaborations, especially in creating new affective technologies that help people with disabilities and with needs for improved emotion communication. Full disclosure: The author is cofounder, chief scientist, and chairman of Affectiva, Inc., a company manufacturing the sensors the author and her students developed in their research, including the sensor mentioned in the autism story above.

About the Authors

Rosalind W. Picard received the Bachelor's degree with highest honors in electrical engineering from the Georgia Institute of Technology, Atlanta, in 1984, and the SM and ScD degrees in electrical engineering and computer science from the Massachusetts Institute of Technology (MIT), Cambridge, in 1986 and 1991, respectively. From 1984 to 1987, she was a member of the technical staff at AT&T Bell Laboratories, Holmdel, New Jersey, where she co-created a new computer architecture for image compression. She interned at Hewlett Packard, IBM, and Scientific Atlanta, and consulted at a variety of companies, including Apple, IRobot, BT, and Motorola. She is currently a professor of media arts and sciences at the MIT Media Laboratory, where she is also the founder and director of the Affective Computing Group, and the leader of a new Autism and Communication Technology initiative. She is co-founder, chairman, and chief scientist of Affectiva, Inc. Dr. Picard has authored or coauthored more than 200 scientific articles. She is best known for pioneering research in image and video content-based retrieval (the original Photobook system), for developing texture models and machine learning for their combination (Society of Models), and for her book Affective Computing (MIT Press, 1997), which envisioned and helped launch the field by that name. Her current research interests include the development of technology to help people comfortably and respectfully measure and communicate affective information, and the development of models of affect that improve decision making and learning.
63 ms
(Ver 3.x)