Digital Human Faces

Guest Editors' Introduction • Catherine Pelachaud and Tamy Boubekeur • July 2010

Theme Articles


This month's theme includes the following featured articles:

The Digital Emily Project:
Achieving a Photoreal Digital Actor
To create and animate a photorealistic digital character, the shape and reflectance of an actress’s face was digitized in 33 different poses. More »

Real-Time Realistic Skin Translucency
A new algorithm renders real-time, photorealistic skin by simulating both reflectance and transmittance of light through a multilayered skin model. More »

Parallel View-Dependent Level-of-Detail Control
Learn about a scheme for view-dependent level-of-detail control that is implemented entirely on programmable graphics hardware. More »

The Expressive Gaze Model: Using Gaze to Express Emotion
The Expressive Gaze Model is a hierarchical framework for composing simple behaviors into emotionally expressive gaze shifts for virtual characters. More »

Modeling Short-Term Dynamics and Variability for Realistic Interactive Facial Animation
A new animation system uses motion models to produce realistic expressive facial motion at interactive speeds. More »

Expressive Facial Animation Synthesis by Learning Speech Coarticulation and Expression Spaces
Take a look at an expressive facial animation synthesis system enabled by automated learning from facial motion capture data. More »

What else is new? »

 

Digital Human FacesFaces are an important communication vector. Through facial expressions, gaze behaviors, and head movements, faces convey information on not only a person's emotional state and attitudes, but also on discursive, pragmatic, and syntactic elements. Expressions result from subtle muscular contractions and wrinkle formations, and we perceive them through a complex filter of subsurface scattering and other nontrivial light reflections. Lately, modeling 3D faces and their expressions has generated a great deal of interest. Researchers cover automatic or interactive generation of 3D geometry as well as rendering and animation techniques.

This research has many applications. One type of application relates to the creation and animation of virtual actors for films and video games. New rendering techniques ensure highly realistic skin models. Animators apply motion capture with or without markers to animate the body and the face. The quality can be precise enough to capture a real actor's performances as well as the slightest movements in emotional expressions.

Another type involves the creation of autonomous agents, particularly embodied conversation agents (ECAs)—autonomous entities with communicative and emotional capabilities. ECAs serve as Web assistants, pedagogical agents, or even companions. Researchers have proposed models to specify and control ECAs' behaviors.

This month's theme covers the broad field of 3D faces and how they're created, rendered, and animated. Moreover, we aim to emphasize the excellent research coming out of the computer graphics and ECA communities.

Selected Articles for Digital Human Faces

In "The Digital Emily Project: Achieving a Photoreal Digital Actor," Oleg Alexander and his colleagues describe a state-of-the-art face performance capture system that produces astonishing visual output. Their system combines some of the most recent capture, rigging, and compositing techniques that provide production-quality special effects. Based on the Light Stage 5 system, their method provides high-resolution animated face geometry, together with accurate measures for specular and subsurface albedo and normals. As a result, this system produces photorealistic animated faces that often cross the well-known "uncanny valley."

Real-time applications such as games also need realistic faces to provide an immersive experience and convey characters' emotions. In "Real-Time Realistic Skin Translucency," Jorge Jimenez and his colleagues propose a scalable approximation method for subsurface scattering. This method produces realistic translucency effects at a high frame rate, requiring minimal extra cost compared to conventional rendering. The final rendering quality approaches that of precomputed pictures and points in an exciting way toward photorealistic real-time rendering of natural shapes.

Again dealing with real-time applications and geared toward an even larger set of applications, Liang Hu and his colleagues propose an efficient GPU implementation of the popular progressive mesh representation in "Parallel View-Dependent Level-of-Detail Control." This algorithm provides a fine-grained parallel update of the displayed level-of-detail, which can be used for adaptive rendering of high-resolution, organic objects, such as faces and other shapes.

In "The Expressive Gaze Model: Using Gaze to Express Emotion," Brent Lance and Stacy Marsella present a model encompassing head, torso, and eye movement in a hierarchical fashion. Their Expressive Gaze Model has two main components: a library of Gaze Warping Transformations and a procedural model of eye movement. These components combine motion capture data, procedural animation, physical animation, and even hand-crafted animation. Lance and Marsella conducted an empirical study to determine the mapping between gaze animation models and emotional states.

Nicolas Stoiber and his colleagues discuss a real-time animation system that reproduces the dynamism of facial expressions in "Modeling Short-Term Dynamics and Variability for Realistic Interactive Facial Animation." Their system uses motion capture data of expressive facial animations. Rather than considering the face as a whole object, the authors develop several motion models, each controlling a given part of the face. These models are trained on the motion capture data and can learn the dynamic characteristics of the various facial parts. Furthermore, a stochastic component ensures reproduction of the variability of human expressions. The resulting animation looks very natural.

Zhigang Deng and his colleagues discuss another example of a learning algorithm applied to motion capture data in "Expressive Facial Animation Synthesis by Learning Speech Coarticulation and Expression Spaces." The algorithm learns expressive speech coarticulation from audio-visual data of an actor speaking sentences with various emotions. The final animation is obtained by blending lip movement from the learned algorithm and facial expressions.

We hope you enjoy reading these articles as much as we did. For further information, take a look at these related links.

 

Catherine PelachaudCatherine Pelachaud is a French National Center for Scientific Research (CNRS) director of research at the Multimedia group in the Signal and Image Processing department at the CNRS Laboratory for Information Processing and Communication, located at Telecom ParisTech. Contact her at catherine.pelachaud@telecom-paristech.fr.

 

 

Tamy BoubekeurTamy Boubekeur is an associate professor of computer science leading the Computer Graphics group in the Signal and Image Processing department at the CNRS Laboratory for Information Processing and Communication, located at Telecom ParisTech. Contact him at tamy.boubekeur@telecom-paristech.fr.

 

 

Article Comments

Please login to post comments.

What's New

Appropriated Interaction Surfaces
We can make small devices easier to use by enabling them to "steal" surface area from the environment. More »

Demythifying Cybersecurity
Cybersecurity myths that recur in popular literature and technical work, such as "more layers of defense are always better than fewer," are debunked. More »

New Products: Family Connections
With family vacations occurring this summer, two iPhone applications help you avoid the "are we there yets?" on long car rides and help record memories of the places you'll visit this summer. More »

ICT R&D Policies: An International Perspective
Information and communication technology (ICT) is a crucial driver of economic—and global—growth. Although the US still performs the most ICT R&D globally, competition has intensified in the past decade. More »

Feature-Ranking Methodology to Diagnose Design-Silicon Timing Mismatch
A machine-learning methodology diagnoses the most important features contributing to mismatches between predicted and actual silicon path lengths. More »

Power7: IBM's Next-Generation Server Processor
The seventh-generation Power processor is IBM's first eight-core processor. Its multicore design scales from 1 to 32 sockets in commercial and scientific environments. More »

Emergence of the Information Technology Discipline
IT has matured rapidly and, thanks to an extremely active and tight-knit academic community, undergone a transition unlike any other computing discipline. More »

Border Security Credibility Assessments via Heterogeneous Sensor Fusion
To help detect deception, this heterogeneous sensor network measures physiological and behavioral indicators of arousal, cognitive effort, and stress while limiting improper profiling. More »

Perspectives on Software Evolution
Barry Boehm says a one-size-fits-all approach to software evolution doesn't work. Kent Beck adds that developers must factor in cost, time, and risk. More »

Status Quo—The Silent Killer
No change should be good news for an IT team, giving them a breather to prepare for upcoming growth, right? Wrong. More »

The World According to Bing
Bing Maps combines route information with aerial and street-level photography to provide a seamless, interactive canvas for visualizing location information. More »

Picture Content Capturing for Mobile Databases
A software environment uses the information from mobile-device sensors to provide rich picture-management functionality, including semantic map personalization. More »

Visualizing a Journal that Serves the Computational Sciences Community
An online article with a VisMashup application illustrates the advantages of an interactive journal over traditional printed formats. More »