A Hierarchical Compositional Model for Face Representation and Sketching
June 2008 (vol. 30 no. 6)
pp. 955-969
DOI Bookmark:
http://doi.ieeecomputersociety.org/10.1109/TPAMI.2008.50
We present a hierarchical-compositional face model as a three-layer And-Or graph to account for the structural variabilities over multiple resolutions. In the And-Or graph, an And-node represents a decomposition of certain graphical structure expanding to a set of Or-nodes with associated relations; an Or-node functions as a switch variable pointing to alternative And-nodes. Faces are represented hierarchically: layer one treats each face as a whole; layer two refines the local facial parts jointly as a set of individual templates; layer three divides face into 15 zones and models facial features like eyecorners or wrinkles. Transitions between layers are realized by measuring the minimum-description-length given the face image complexity. Diverse face representations are formed by drawing from hierarchical dictionaries of faces, parts and skin features. A sketch captures the most informative part of a face in a concise and potentially robust representation. However, generating good facial sketches is challenging because of the rich facial details and large structural variations, especially in the high-resolution images. The representing power of our generative model is demostrated by reconstructing high-resolution face images and generating cartoon sketches. Our model is useful for applications such as face recognition, non-photorealisitc rendering, super-esolution, and low-bit rate face coding.
[1] S.P. Abney, “Stochastic Attribute-Value Grammars,” Computational Linguistics, vol. 23, no. 4, pp. 597-618, 1997.[2] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.[3] S. Baker and T. Kanade, “Hallucinating Faces,” Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2000.[4] V. Bruce, E. Hanna, N. Dench, P. Healey, and M. Burton, “The importance of ‘Mass’ in Line Drawings of Faces,” Applied Cognitive Psychology, vol. 6, pp. 619-628, 1992.[5] H. Chen, Y.Q. Xu, H.Y. Shum, S.C. Zhu, and N.N. Zhen, “Example-Based Facial Sketch Generation with Non-Parametric Sampling,” Proc. IEEE Int'l Conf. Computer Vision, 2001.[6] H. Chen, Z.J. Xu, Z.Q. Liu, and S.C. Zhu, “Composite Templates for Cloth Modeling and Sketching,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.[7] T.F. Cootes, C.J. Taylor, D. Cooper, and J. Graham, “Active Shape Models—Their Training and Applications,” Computer Vision and Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.[8] T.F. Cootes and C.J. Taylor, “Constrained Active Appearance Models,” Proc. IEEE Int'l Conf. Computer Vision, 2001.[9] R.H. Davies, T.F. Cootes, C. Twining, and C.J. Taylor, “An Information Theoretic Approach to Statistical Shape Modelling,” Proc. British Machine Vision Conf., 2001.[10] M. Fischler and R. Elschlager, “The Representation and Matching of Pictorial Structures,” IEEE Trans. Computers, vol. 22, no. 1, p.67C92, Jan. 1973.[11] K.S. Fu, Syntactic Pattern Recognition and Applications. Prentice Hall, 1981.[12] C. Guo, S.C. Zhu, and Y.N. Wu, “Towards a Mathematical Theory of Primal Sketch and Sketchability,” Proc. IEEE Int'l Conf. Computer Vision, 2003.[13] P.L. Hallinan, G.G. Gordon, A.L. Yuille, and D.B. Mumford, Two and Three Dimensional Patterns of the Face. A.K. Peters, 1999.[14] B. Heisele, P. Ho, J. Wu, and T. Poggio, “Face Recognition: Component-Based versus Global Approaches,” Computer Vision and Image Understanding, vol. 91, nos. 1/2, pp. 6-21, 2003.[15] G. Hua and Y. Wu, “Multi-Scale Visual Tracking by Sequential Belief Propagation,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2004.[16] M.J. Jones and T. Poggio, “Multi-Dimensional Morphable Models: A Framework for Representing and Matching Object Classes,” Int'l J. Computer Vision, vol. 2, no. 29, pp. 107-131, 1998.[17] T. Kanade, Computer Recognition of Human Faces, 1973.[18] H. Koshimizu, M. Tominaga, T. Fujiwara, and K. Murakami, “On Kansei Facial Processing for Computerized Caricaturing System Picasso,” Proc. Int'l Conf. Systems, Man, and Cybernetics, vol. 6, pp.294-299, 1999.[19] L. Liang, F. Wen, Y.Q. Xu, X. Tang, and H.Y. Shum, “Accurate Face Alignment Using Shape Constrained Markov Network,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.[20] T. Lindeberg, Scale-Space Theory in Computer Vision. Kluwer Academic, 1994.[21] C. Liu, H.Y. Shum, and C.S. Zhang, “Hierarchical Shape Model for Automatic Face Localization,” Proc. European Conf. Computer Vision, pp. 687-703, 2002.[22] C. Liu, H.Y. Shum, and C.S. Zhang, “Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2001.[23] A.M. Martinez and R. Benavente, “The AR Face Database,” CVC Technical Report 24, 1998.[24] J. Pearl, Heuristics: Intelligent Search Strategies for Computer Problem Solving. Addison-Wesley, 1984.[25] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.[26] A. Pentland, B. Moghaddam, and T. Starner, “View-Based and Modular Eigenspaces for Face Recognition,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1994.[27] P.J. Phillips, H. Wechsler, J. Huang, and P. Rauss, “The FERET Database and Evaluation Procedure for Face Recognition Algorithms,” Image and Vision Computing J., vol. 16, no. 5, pp. 295-306, 1998.[28] J. Rekers and A. Schürr, “A Parsing Algorithm for Context Sensitive Graph Grammars,” technical report, Leiden Univ., 1995.[29] X. Tang and X. Wang, “Face Sketch Synthesis and Recognition,” Proc. IEEE Int'l Conf. Computer Vision, 2003.[30] Y. Tian, T. Kanade, and J. Cohn, “Recognizing Action Units of Facial Expression Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 229-234, Feb. 2001.[31] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J.Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.[32] S. Ullman and E. Sali, “Object Classification Using a Fragment-Based Representation,” Proc. British Machine Vision Conf., 2000.[33] P. Viola and M. Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2001.[34] M. Weber, M. Welling, and P. Perona, “Towards Automatic Discovery of Object Categories,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2000.[35] J. Xiao, S. Baker, and T. Kanade, “Real-Time Combined 2D$+$ 3D Active Appearance Models,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2004.[36] Z.J. Xu, H. Chen, and S.C. Zhu, “A High Resolution Gramatical Model for Face Representation and Sketching,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2005.[37] Z.J. Xu and J. Luo, “Face Recognition by Expression-Driven Sketch Graph Matching,” Proc. Int'l Conf. Pattern Recognition, 2006.[38] M.H. Yang, D.J. Kriegman, and N. Ahuja, “Detecting Faces in Images: A Survey,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 1-25, Jan. 2002.[39] Z.Y. Yao, X. Yang, and S.C. Zhu, “Introduction to a Large Scale General Purpose Groundtruth Dataset: Methodology, Annotation Tool, and Benchmarks,” Proc. Int'l Workshop Energy Minimization Methods in Computer Vision and Pattern Recognition, 2007.[40] A.L. Yuille, D. Cohen, and P. Hallinan, “Feature Extraction from Faces Using Deformable Templates,” Int'l J. Computer Vision, vol. 8, pp. 99-111, 1992.[41] W. Zhao, R. Chellappa, A. Rosenfeld, and P.J. Phillips, “Face Recognition: A Literature Survey,” UMD Cfar Technical Report 948, 2000.[42] S.C. Zhu, Y.N. Wu, and D.B. Mumford, “Filters, Random Fields and Maximum Entropy (FRAME),” Int'l J. Computer Vision, vol. 27, no. 2, pp. 1-20, 1998.[43] S.C. Zhu and D. Mumford, “Quest for a Stochastic Grammar of Images,” Foundations and Trends in Computer Graphics and Vision, 2007.[44] Z. Xu and J. Luo, “Accurate Dynamic Sketching of Faces from Video,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, Workshop on Semantic Learning Applications in Multimedia, 2007.
Index Terms:
Image Processing and Computer Vision, Hierarchical, Statistical
Citation:
Zijian Xu, Hong Chen, Song-Chun Zhu, Jiebo Luo, "A Hierarchical Compositional Model for Face Representation and Sketching," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 6, pp. 955-969, Feb. 2008, doi:10.1109/TPAMI.2008.50
Usage of this product signifies your acceptance of the
Terms of Use.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb