This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Hierarchical Compositional Model for Face Representation and Sketching
June 2008 (vol. 30 no. 6)
pp. 955-969
Zijian Xu, Moody's Corporation, Wall Street Analytics
Hong Chen, Brion Technologies Incorporated
Song-Chun Zhu, University of California, Los Angeles, Los Angeles
Jiebo Luo, Kodak Research Labs, Rochester
We present a hierarchical-compositional face model as a three-layer And-Or graph to account for the structural variabilities over multiple resolutions. In the And-Or graph, an And-node represents a decomposition of certain graphical structure expanding to a set of Or-nodes with associated relations; an Or-node functions as a switch variable pointing to alternative And-nodes. Faces are represented hierarchically: layer one treats each face as a whole; layer two refines the local facial parts jointly as a set of individual templates; layer three divides face into 15 zones and models facial features like eyecorners or wrinkles. Transitions between layers are realized by measuring the minimum-description-length given the face image complexity. Diverse face representations are formed by drawing from hierarchical dictionaries of faces, parts and skin features. A sketch captures the most informative part of a face in a concise and potentially robust representation. However, generating good facial sketches is challenging because of the rich facial details and large structural variations, especially in the high-resolution images. The representing power of our generative model is demostrated by reconstructing high-resolution face images and generating cartoon sketches. Our model is useful for applications such as face recognition, non-photorealisitc rendering, super-esolution, and low-bit rate face coding.

[1] S.P. Abney, “Stochastic Attribute-Value Grammars,” Computational Linguistics, vol. 23, no. 4, pp. 597-618, 1997.
[2] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[3] S. Baker and T. Kanade, “Hallucinating Faces,” Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2000.
[4] V. Bruce, E. Hanna, N. Dench, P. Healey, and M. Burton, “The importance of ‘Mass’ in Line Drawings of Faces,” Applied Cognitive Psychology, vol. 6, pp. 619-628, 1992.
[5] H. Chen, Y.Q. Xu, H.Y. Shum, S.C. Zhu, and N.N. Zhen, “Example-Based Facial Sketch Generation with Non-Parametric Sampling,” Proc. IEEE Int'l Conf. Computer Vision, 2001.
[6] H. Chen, Z.J. Xu, Z.Q. Liu, and S.C. Zhu, “Composite Templates for Cloth Modeling and Sketching,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.
[7] T.F. Cootes, C.J. Taylor, D. Cooper, and J. Graham, “Active Shape Models—Their Training and Applications,” Computer Vision and Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.
[8] T.F. Cootes and C.J. Taylor, “Constrained Active Appearance Models,” Proc. IEEE Int'l Conf. Computer Vision, 2001.
[9] R.H. Davies, T.F. Cootes, C. Twining, and C.J. Taylor, “An Information Theoretic Approach to Statistical Shape Modelling,” Proc. British Machine Vision Conf., 2001.
[10] M. Fischler and R. Elschlager, “The Representation and Matching of Pictorial Structures,” IEEE Trans. Computers, vol. 22, no. 1, p.67C92, Jan. 1973.
[11] K.S. Fu, Syntactic Pattern Recognition and Applications. Prentice Hall, 1981.
[12] C. Guo, S.C. Zhu, and Y.N. Wu, “Towards a Mathematical Theory of Primal Sketch and Sketchability,” Proc. IEEE Int'l Conf. Computer Vision, 2003.
[13] P.L. Hallinan, G.G. Gordon, A.L. Yuille, and D.B. Mumford, Two and Three Dimensional Patterns of the Face. A.K. Peters, 1999.
[14] B. Heisele, P. Ho, J. Wu, and T. Poggio, “Face Recognition: Component-Based versus Global Approaches,” Computer Vision and Image Understanding, vol. 91, nos. 1/2, pp. 6-21, 2003.
[15] G. Hua and Y. Wu, “Multi-Scale Visual Tracking by Sequential Belief Propagation,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2004.
[16] M.J. Jones and T. Poggio, “Multi-Dimensional Morphable Models: A Framework for Representing and Matching Object Classes,” Int'l J. Computer Vision, vol. 2, no. 29, pp. 107-131, 1998.
[17] T. Kanade, Computer Recognition of Human Faces, 1973.
[18] H. Koshimizu, M. Tominaga, T. Fujiwara, and K. Murakami, “On Kansei Facial Processing for Computerized Caricaturing System Picasso,” Proc. Int'l Conf. Systems, Man, and Cybernetics, vol. 6, pp.294-299, 1999.
[19] L. Liang, F. Wen, Y.Q. Xu, X. Tang, and H.Y. Shum, “Accurate Face Alignment Using Shape Constrained Markov Network,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.
[20] T. Lindeberg, Scale-Space Theory in Computer Vision. Kluwer Academic, 1994.
[21] C. Liu, H.Y. Shum, and C.S. Zhang, “Hierarchical Shape Model for Automatic Face Localization,” Proc. European Conf. Computer Vision, pp. 687-703, 2002.
[22] C. Liu, H.Y. Shum, and C.S. Zhang, “Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2001.
[23] A.M. Martinez and R. Benavente, “The AR Face Database,” CVC Technical Report 24, 1998.
[24] J. Pearl, Heuristics: Intelligent Search Strategies for Computer Problem Solving. Addison-Wesley, 1984.
[25] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
[26] A. Pentland, B. Moghaddam, and T. Starner, “View-Based and Modular Eigenspaces for Face Recognition,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1994.
[27] P.J. Phillips, H. Wechsler, J. Huang, and P. Rauss, “The FERET Database and Evaluation Procedure for Face Recognition Algorithms,” Image and Vision Computing J., vol. 16, no. 5, pp. 295-306, 1998.
[28] J. Rekers and A. Schürr, “A Parsing Algorithm for Context Sensitive Graph Grammars,” technical report, Leiden Univ., 1995.
[29] X. Tang and X. Wang, “Face Sketch Synthesis and Recognition,” Proc. IEEE Int'l Conf. Computer Vision, 2003.
[30] Y. Tian, T. Kanade, and J. Cohn, “Recognizing Action Units of Facial Expression Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 229-234, Feb. 2001.
[31] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J.Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
[32] S. Ullman and E. Sali, “Object Classification Using a Fragment-Based Representation,” Proc. British Machine Vision Conf., 2000.
[33] P. Viola and M. Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2001.
[34] M. Weber, M. Welling, and P. Perona, “Towards Automatic Discovery of Object Categories,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2000.
[35] J. Xiao, S. Baker, and T. Kanade, “Real-Time Combined 2D$+$ 3D Active Appearance Models,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2004.
[36] Z.J. Xu, H. Chen, and S.C. Zhu, “A High Resolution Gramatical Model for Face Representation and Sketching,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[37] Z.J. Xu and J. Luo, “Face Recognition by Expression-Driven Sketch Graph Matching,” Proc. Int'l Conf. Pattern Recognition, 2006.
[38] M.H. Yang, D.J. Kriegman, and N. Ahuja, “Detecting Faces in Images: A Survey,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 1-25, Jan. 2002.
[39] Z.Y. Yao, X. Yang, and S.C. Zhu, “Introduction to a Large Scale General Purpose Groundtruth Dataset: Methodology, Annotation Tool, and Benchmarks,” Proc. Int'l Workshop Energy Minimization Methods in Computer Vision and Pattern Recognition, 2007.
[40] A.L. Yuille, D. Cohen, and P. Hallinan, “Feature Extraction from Faces Using Deformable Templates,” Int'l J. Computer Vision, vol. 8, pp. 99-111, 1992.
[41] W. Zhao, R. Chellappa, A. Rosenfeld, and P.J. Phillips, “Face Recognition: A Literature Survey,” UMD Cfar Technical Report 948, 2000.
[42] S.C. Zhu, Y.N. Wu, and D.B. Mumford, “Filters, Random Fields and Maximum Entropy (FRAME),” Int'l J. Computer Vision, vol. 27, no. 2, pp. 1-20, 1998.
[43] S.C. Zhu and D. Mumford, “Quest for a Stochastic Grammar of Images,” Foundations and Trends in Computer Graphics and Vision, 2007.
[44] Z. Xu and J. Luo, “Accurate Dynamic Sketching of Faces from Video,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, Workshop on Semantic Learning Applications in Multimedia, 2007.

Index Terms:
Image Processing and Computer Vision, Hierarchical, Statistical
Citation:
Zijian Xu, Hong Chen, Song-Chun Zhu, Jiebo Luo, "A Hierarchical Compositional Model for Face Representation and Sketching," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 6, pp. 955-969, June 2008, doi:10.1109/TPAMI.2008.50
Usage of this product signifies your acceptance of the Terms of Use.