Pages: pp. 1921-1924
In recent years, there has been an emerging demand for robust face recognition algorithms that are able to deal with real-world face images. This is largely due to two factors. First, consumers have shown an increasing desire to annotate or tag their digital photos to facilitate organization, access, and online sharing of their personal albums. Industrial responses to this consumer desire can be exemplifed by successful commercial face recognition systems included in applications and web sites such as Google Picasa, Windows Live Photo Gallery, Apple iPhoto, face.com, PolarRose, etc. Second, the growing applications in public security also call for robust face recognition technologies that can identify individuals from surveillance cameras in uncontrolled situations.
In consumer digital imaging, face recognition must contend with uncontrolled lighting, large pose variations, a range of facial expressions, make-up, changes in facial hair, eyewear, weight gain, aging, and partial occlusions. Similarly, in scenarios such as visual surveillance, videos are often acquired in uncontrolled situations or from moving cameras. These factors have been the focus of face recognition research for decades, but they are still not well resolved.
In addition to these market forces, face recognition also represents an ongoing set of key scientific challenges. How can we match or exceed the performance of humans on face on many real-world face recognition tasks (e.g., recognition of people known to the human viewer)? How can we learn a good model of a face from a small number of examples? How can we achieve the level of robustness exhibited by human face recognition? These questions and new applications for the technology promise to keep this area active for the forseeable future.
Standard face recognition systems often start with a set of labeled gallery faces. When a new probe image is provided, it is matched against the gallery faces to be recognized as a known face or rejected. In a well-controlled setting, face images can be carefully captured for both gallery and probe faces. In a moderately controlled setting, we may have quality control over either the gallery faces or the probe faces, but not both. In an uncontrolled setting, we lose control of both.
Face recognition in well-controlled settings has been extensively studied and is relatively mature. Earlier face recognition methods often directly appled pattern recognition and machine learning techniques on informative face features and are only effective when the probe and gallery images are frontal. More recently, to achieve higher recognition performance, many works have started to consider more precise geometric, shape, lighting, and reflectance models of faces. Notwithstanding the successes of these techniques, there remains much room for improvement of face recognition in real-world scenarios since, overall, much less attention has been paid to these less controlled settings.
Two general misconceptions are that face recognition is a solved problem and, on the other hand, that the uncontrolled scenarios are too difficult to address in practice. Neither is true in our opinion. A primary purpose of this special section is to deliver the following message to the community: Although significant progress has been made in the last few decades, there remain plenty of challenges and opportunities ahead.
In the consumer digital imaging domain, practical face annotation systems are emerging based on existing face rec-ognition technologies. This brings human factors in the assistanted annotation system since good user interface (UI) and user experience (UX) design are essential in order to compensate for possible failures of the face recognition algorithm. Last but not least, in many of these applications there may be rich additional contextual information and meta-data that one can leverage to improve face recognition. A multidisciplinary exploration may be required to deliver real working systems.
The motivations for organizing this special section were to better address the challenges of face recognition in real-world scenarios, to promote systematic research and evaluation of promising methods and systems, to provide a snapshot of where we are in this domain, and to stimulate discussion about future directions. We solicited original contributions of research on all aspects of real-world face recognition, including:
We received 42 original submissions, four of which were rejected without review; the other 38 papers entered the normal review process. Each paper was reviewed by three reviewers who are experts in their respective topics. More than 100 expert reviewers have been involved in the review process.
The papers were equally distributed among the guest editors. A final decision for each paper was made by at least two guest editors assigned to it. To avoid conflict of interest, no guest editor submitted any papers to this special section.
Six papers were accepted through the rigorous review process, for an overall acceptance rate of 14.3 percent. The accepted papers can be placed into three categories: face recognition in real-world watch-list visual surveillance systems, 3D modeling for pose variant face recognition, and design of robust face similarity features and metrics for face recognition in consumer photos. In the following, we briefly summarize the papers in each category.
In “Toward Development of a Face Recognition System for Watch-List Surveillance,” Kamgar-Parsi et al. examine the problem of designing a face recognition system for a watch-list visual surveillance system where a small set of people needs to be identified from a large number of people passing through surveillance cameras. Their approach is to use view morphing to automatically generate borderline faces to define the face space of a person, and then to train classifiers based on the borderline faces from each person in the watch-list. The method attacks a real-world face recognition problem with an interesting and solid approach.
3D face recognition has been regarded as a natural solution to pose variation. In “Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition,” Passalis et al. propose using facial symmetry to handle pose variation in 3D face recognition, while in “Unconstrained Pose Invariant Face Recognition Using 3D Generic Elastic Models,” Prabhu et al. propose a generic 3D elastic model for pose invariant face recognition. Both are plausible approaches for using 3D information to assist in face recognition under large pose variations. While historically 3D face recognition has been criticized for lack of real-world 3D sensory cameras, this issue may be resolved in the future with inexpensive 3D sensors as evidenced by the PrimeSense sensor used in the Xbox Kinect from Microsoft.
The other three accepted papers all deal with face recognition in photos and images in the wild. In “Describable Visual Attributes for Face Verification and Image Search,” Kumar et al. present a face verification algorithm based on a representation that uses a set of describable attributes. Their classifier achieves very good results on two publicly available benchmarks, namely Labeled Faces in the Wild (LFW) and the Public Figures (PubFig) data set. Since their method was first published at ICCV '09, there has been a lot of work using attribute-based representations for various visual recognition problems beyond face recognition.
In “Effective Unconstrained Face Recognition by Combining Multiple Descriptors and Learned Background Statistics,” Wolf et al. propose an approach for face verification in the wild that combines multiple descriptors with learned statistics from background context. Its face verification accuracy ranked first on the LFW benchmark. In “Scalable Face Image Retrieval with Identity-Based Quantization and Multireference Reranking,” Wu et al. present a scalable face image retrieval system using identity-based quantization to build a visual representation and multiple references for reranking. It builds a good foundation to tackle the problem of searching for face images over the internet image corpus.
The face recognition research community has built a variety of solid benchmarks to evaluate different algorithms. It is vital for researchers to leverage these databases to conduct solid and convincing experimental validation and compare with the state-of-the-art. Yet there also comes a time when performance on a benchmark reaches ceiling performance or methods become overengineered for nuances of a data set, and modest performance gains may be indicative of overfitting.
Alternatively, some new works or operational scenarios may push the envelope in directions that are not well re-presented with existing benchmarks; in such cases, authors may need to develop alternative benchmarks and justify this need in subsequent publications. Interestingly, real-world face recognition methods that achieve state-of-the-art performance on data sets like LFW may actually perform worse on constrained, frontal data sets like FERET. We should not be surprised by this, and we should embrace methods for where they are effective.
Through the editorial process of this special section, it has been our observation that the joint efforts of the whole face recognition research community have made many applications of real-world face recognition achievable, but there are still many challenges to address and opportunities to explore.
Claims that face recognition is a solved problem are overly bold and optimistic. On the contrary, claims that face recognition in real-world scenarios is next to impossible are simply too pessimistic, given the success of the aforemen-tioned commercial face recognition systems. We hope this special section on Real-World Face Recognition will serve as a reference point toward an objective evaluation of the community's progress on face recognition research.
We thank all authors for their enthusiastic contributions to this special section. We also thank the 100+ reviewers for their thoughtful reviews of the submissions, which were vital to ensure the quality of this special section. We greatly appreciate Editor-in-Chief Professor Ramin Zabih's timely help in resolving many issues that occurred during the review process. Last but not least, we thank Andy Morton for his help in all aspect of the editorial administration.
David J. Kriegman
Thomas S. Huang