The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - November (2009 vol.31)
pp: 1955-1967
Xiaogang Wang , The Chinese University of Hong Kong, Hong Kong
Xiaoou Tang , The Chinese University of Hong Kong, Hong Kong
ABSTRACT
In this paper, we propose a novel face photo-sketch synthesis and recognition method using a multiscale Markov Random Fields (MRF) model. Our system has three components: 1) given a face photo, synthesizing a sketch drawing; 2) given a face sketch drawing, synthesizing a photo; and 3) searching for face photos in the database based on a query sketch drawn by an artist. It has useful applications for both digital entertainment and law enforcement. We assume that faces to be studied are in a frontal pose, with normal lighting and neutral expression, and have no occlusions. To synthesize sketch/photo images, the face region is divided into overlapping patches for learning. The size of the patches decides the scale of local face structures to be learned. From a training set which contains photo-sketch pairs, the joint photo-sketch model is learned at multiple scales using a multiscale MRF model. By transforming a face photo to a sketch (or transforming a sketch to a photo), the difference between photos and sketches is significantly reduced, thus allowing effective matching between the two in face sketch recognition. After the photo-sketch transformation, in principle, most of the proposed face photo recognition approaches can be applied to face sketch recognition in a straightforward way. Extensive experiments are conducted on a face sketch database including 606 faces, which can be downloaded from our Web site (http://mmlab.ie.cuhk.edu.hk/facesketch.html).
INDEX TERMS
Face recognition, face sketch synthesis, face sketch recognition, multiscale Markov random field.
CITATION
Xiaogang Wang, Xiaoou Tang, "Face Photo-Sketch Synthesis and Recognition", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 11, pp. 1955-1967, November 2009, doi:10.1109/TPAMI.2008.222
REFERENCES
[1] R.G. Uhl and N.D.V. Lobo, “A Framework for Recognizing a Facial Image from a Police Sketch,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1996.
[2] W. Konen, “Comparing Facial Line Drawings with Gray-Level Images: A Case Study on Phantomas,” Proc. Int'l Conf. Artificial Neural Networks, 1996.
[3] X. Tang and X. Wang, “Face Sketch Recognition,” IEEE Trans. Circuits and Systems for Video Technology, vol. 14, no. 1, pp. 50-57, Jan. 2004.
[4] X. Tang and X. Wang, “Face Sketch Synthesis and Recognition,” Proc. IEEE Int'l Conf. Computer Vision, 2003.
[5] Q. Liu, X. Tang, H. Jin, H. Lu, and S. Ma, “A Nonlinear Approach for Face Sketch Synthesis and Recognition,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[6] J. Zhong, X. Gao, and C. Tian, “Face Sketch Synthesis Using a E-Hmm and Selective Ensemble,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, 2007.
[7] X. Gao, J. Zhong, and C. Tian, “Sketch Synthesis Algorithm Based on E-Hmm and Selective Ensemble,” IEEE Trans. Circuits and Systems for Video Technology, vol. 18, no. 4, pp. 487-496, Apr. 2008.
[8] H. Koshimizu, M. Tominaga, T. Fujiwara, and K. Murakami, “On Kansei Facial Processing for Computerized Facial Caricaturing System Picasso,” Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, 1999.
[9] S. Iwashita, Y. Takeda, and T. Onisawa, “Expressive Facial Caricature Drawing,” Proc. IEEE Int'l Conf. Fuzzy Systems, 1999.
[10] J. Benson and D.I. Perrett, “Perception and Recognition of Photographic Quality Facial Caricatures: Implications for the Recognition of Natural Images,” European J. Cognitive Psychology, vol. 3, pp. 105-135, 1991.
[11] V. Bruce, E. Hanna, N. Dench, P. Healy, and A.M. Burton, “The Importance of Mass in Line Drawings of Faces,” Applied Cognitive Psychology, vol. 6, pp. 619-628, 1992.
[12] V. Bruce and G.W. Humphreys, “Recognizing Objects and Faces,” Visual Cognition, vol. 1, pp. 141-180, 1994.
[13] G.M. Davies, H.D. Ellis, and J.W. Shepherd, “Face Recognition Accuracy As a Function of Mode of Representation,” J. Applied Psychology, vol. 63, pp. 180-187, 1978.
[14] G. Rhodes and T. Tremewan, “Understanding Face Recognition: Caricature Effects, Inversion, and the Homogeneity Problem,” Visual Cognition, vol. 1, pp. 275-311, 1994.
[15] W.T. Freeman, J.B. Tenenbaum, and E. Pasztor, “An Example-Based Approach to Style Translation for Line Drawings,” technical report, MERL, 1999.
[16] H. Chen, Y. Xu, H. Shum, S. Zhu, and N. Zheng, “Example-Based Facial Sketch Generation with Non-Parametric Sampling,” Proc. IEEE Int'l Conf. Computer Vision, 2001.
[17] T.F. Cootes, G.J. Edwards, and C.J. Taylor, “Active Appearance Model,” Proc. European Conf. Computer Vision, 1998.
[18] W.T. Freeman, E.C. Pasztor, and O.T. Carmichael, “Learning Low-Level Vision,” Int'l J. Computer Vision, vol. 40, pp. 25-47, 2000.
[19] P. Viola and M. Jones, “Real-Time Object Detection,” Int'l J. Computer Vision, vol. 52, pp. 137-154, 2004.
[20] J.S. Yedidia, W.T. Freeman, and Y. Weiss, “Understanding Belief Propagation and Its Generalizations,” Exploring Artificial Intelligence in the New Millennium, Morgan Kaufmann, 2003.
[21] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization Via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[22] A.A. Efros and W.T. Freeman, “Quilting for Texture Synthesis and Transfer,” Proc. ACM Conf. Computer Graphics and Interactive Techniques, 2001.
[23] A.M. Martinez and R. Benavente, “The AR Face Database,” Technical Report 24, CVC, 1998.
[24] K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre, “Xm2vtsdb: The Extended of m2vts Database,” Proc. Int'l Conf. Audio- and Video-Based Person Authentication, 1999.
[25] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J.Cognitive Neuroscience, vol. 3, pp. 71-86, 1991.
[26] B. Moghaddam and A. Pentland, “Probabilistic Visual Learning for Object Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 696-710, July 1997.
[27] P.N. Belhumeur, J. Hespanda, and D. Kiregeman, “Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[28] L. Chen, H. Liao, M. Ko, J. Lin, and G. Yu, “A New Lda Based Face Recognition System which Can Solve the Small Sample Size Problem,” Pattern Recognition, vol. 33, pp. 1713-1726, 2000.
[29] X. Wang and X. Tang, “Dual-Space Linear Discriminant Analysis for Face Recognition,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2004.
[30] X. Wang and X. Tang, “Random Sampling Lda for Face Recognition,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2004.
[31] X. Wang and X. Tang, “Random Sampling for Subspace Face Recognition,” Int'l J. Computer Vision, vol. 70, pp. 91-104, 2006.
[32] L. Wiskott, J. Fellous, N. Kruger, and C. Malsburg, “Face Recognition by Elastic Bunch Graph Matching,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 775-779, July 1997.
18 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool