The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.06 - June (2008 vol.30)
pp: 955-969
Zijian Xu , Moody's Corporation, Wall Street Analytics
Hong Chen , Brion Technologies Incorporated
Song-Chun Zhu , University of California, Los Angeles, Los Angeles
Jiebo Luo , Kodak Research Labs, Rochester
ABSTRACT
We present a hierarchical-compositional face model as a three-layer And-Or graph to account for the structural variabilities over multiple resolutions. In the And-Or graph, an And-node represents a decomposition of certain graphical structure expanding to a set of Or-nodes with associated relations; an Or-node functions as a switch variable pointing to alternative And-nodes. Faces are represented hierarchically: layer one treats each face as a whole; layer two refines the local facial parts jointly as a set of individual templates; layer three divides face into 15 zones and models facial features like eyecorners or wrinkles. Transitions between layers are realized by measuring the minimum-description-length given the face image complexity. Diverse face representations are formed by drawing from hierarchical dictionaries of faces, parts and skin features. A sketch captures the most informative part of a face in a concise and potentially robust representation. However, generating good facial sketches is challenging because of the rich facial details and large structural variations, especially in the high-resolution images. The representing power of our generative model is demostrated by reconstructing high-resolution face images and generating cartoon sketches. Our model is useful for applications such as face recognition, non-photorealisitc rendering, super-esolution, and low-bit rate face coding.
INDEX TERMS
Image Processing and Computer Vision, Hierarchical, Statistical
CITATION
Zijian Xu, Hong Chen, Song-Chun Zhu, Jiebo Luo, "A Hierarchical Compositional Model for Face Representation and Sketching", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 6, pp. 955-969, June 2008, doi:10.1109/TPAMI.2008.50
REFERENCES
[1] S.P. Abney, “Stochastic Attribute-Value Grammars,” Computational Linguistics, vol. 23, no. 4, pp. 597-618, 1997.
[2] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[3] S. Baker and T. Kanade, “Hallucinating Faces,” Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, 2000.
[4] V. Bruce, E. Hanna, N. Dench, P. Healey, and M. Burton, “The importance of ‘Mass’ in Line Drawings of Faces,” Applied Cognitive Psychology, vol. 6, pp. 619-628, 1992.
[5] H. Chen, Y.Q. Xu, H.Y. Shum, S.C. Zhu, and N.N. Zhen, “Example-Based Facial Sketch Generation with Non-Parametric Sampling,” Proc. IEEE Int'l Conf. Computer Vision, 2001.
[6] H. Chen, Z.J. Xu, Z.Q. Liu, and S.C. Zhu, “Composite Templates for Cloth Modeling and Sketching,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.
[7] T.F. Cootes, C.J. Taylor, D. Cooper, and J. Graham, “Active Shape Models—Their Training and Applications,” Computer Vision and Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.
[8] T.F. Cootes and C.J. Taylor, “Constrained Active Appearance Models,” Proc. IEEE Int'l Conf. Computer Vision, 2001.
[9] R.H. Davies, T.F. Cootes, C. Twining, and C.J. Taylor, “An Information Theoretic Approach to Statistical Shape Modelling,” Proc. British Machine Vision Conf., 2001.
[10] M. Fischler and R. Elschlager, “The Representation and Matching of Pictorial Structures,” IEEE Trans. Computers, vol. 22, no. 1, p.67C92, Jan. 1973.
[11] K.S. Fu, Syntactic Pattern Recognition and Applications. Prentice Hall, 1981.
[12] C. Guo, S.C. Zhu, and Y.N. Wu, “Towards a Mathematical Theory of Primal Sketch and Sketchability,” Proc. IEEE Int'l Conf. Computer Vision, 2003.
[13] P.L. Hallinan, G.G. Gordon, A.L. Yuille, and D.B. Mumford, Two and Three Dimensional Patterns of the Face. A.K. Peters, 1999.
[14] B. Heisele, P. Ho, J. Wu, and T. Poggio, “Face Recognition: Component-Based versus Global Approaches,” Computer Vision and Image Understanding, vol. 91, nos. 1/2, pp. 6-21, 2003.
[15] G. Hua and Y. Wu, “Multi-Scale Visual Tracking by Sequential Belief Propagation,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2004.
[16] M.J. Jones and T. Poggio, “Multi-Dimensional Morphable Models: A Framework for Representing and Matching Object Classes,” Int'l J. Computer Vision, vol. 2, no. 29, pp. 107-131, 1998.
[17] T. Kanade, Computer Recognition of Human Faces, 1973.
[18] H. Koshimizu, M. Tominaga, T. Fujiwara, and K. Murakami, “On Kansei Facial Processing for Computerized Caricaturing System Picasso,” Proc. Int'l Conf. Systems, Man, and Cybernetics, vol. 6, pp.294-299, 1999.
[19] L. Liang, F. Wen, Y.Q. Xu, X. Tang, and H.Y. Shum, “Accurate Face Alignment Using Shape Constrained Markov Network,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.
[20] T. Lindeberg, Scale-Space Theory in Computer Vision. Kluwer Academic, 1994.
[21] C. Liu, H.Y. Shum, and C.S. Zhang, “Hierarchical Shape Model for Automatic Face Localization,” Proc. European Conf. Computer Vision, pp. 687-703, 2002.
[22] C. Liu, H.Y. Shum, and C.S. Zhang, “Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2001.
[23] A.M. Martinez and R. Benavente, “The AR Face Database,” CVC Technical Report 24, 1998.
[24] J. Pearl, Heuristics: Intelligent Search Strategies for Computer Problem Solving. Addison-Wesley, 1984.
[25] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
[26] A. Pentland, B. Moghaddam, and T. Starner, “View-Based and Modular Eigenspaces for Face Recognition,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1994.
[27] P.J. Phillips, H. Wechsler, J. Huang, and P. Rauss, “The FERET Database and Evaluation Procedure for Face Recognition Algorithms,” Image and Vision Computing J., vol. 16, no. 5, pp. 295-306, 1998.
[28] J. Rekers and A. Schürr, “A Parsing Algorithm for Context Sensitive Graph Grammars,” technical report, Leiden Univ., 1995.
[29] X. Tang and X. Wang, “Face Sketch Synthesis and Recognition,” Proc. IEEE Int'l Conf. Computer Vision, 2003.
[30] Y. Tian, T. Kanade, and J. Cohn, “Recognizing Action Units of Facial Expression Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 229-234, Feb. 2001.
[31] M. Turk and A. Pentland, “Eigenfaces for Recognition,” J.Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
[32] S. Ullman and E. Sali, “Object Classification Using a Fragment-Based Representation,” Proc. British Machine Vision Conf., 2000.
[33] P. Viola and M. Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2001.
[34] M. Weber, M. Welling, and P. Perona, “Towards Automatic Discovery of Object Categories,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2000.
[35] J. Xiao, S. Baker, and T. Kanade, “Real-Time Combined 2D$+$ 3D Active Appearance Models,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2004.
[36] Z.J. Xu, H. Chen, and S.C. Zhu, “A High Resolution Gramatical Model for Face Representation and Sketching,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[37] Z.J. Xu and J. Luo, “Face Recognition by Expression-Driven Sketch Graph Matching,” Proc. Int'l Conf. Pattern Recognition, 2006.
[38] M.H. Yang, D.J. Kriegman, and N. Ahuja, “Detecting Faces in Images: A Survey,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 1-25, Jan. 2002.
[39] Z.Y. Yao, X. Yang, and S.C. Zhu, “Introduction to a Large Scale General Purpose Groundtruth Dataset: Methodology, Annotation Tool, and Benchmarks,” Proc. Int'l Workshop Energy Minimization Methods in Computer Vision and Pattern Recognition, 2007.
[40] A.L. Yuille, D. Cohen, and P. Hallinan, “Feature Extraction from Faces Using Deformable Templates,” Int'l J. Computer Vision, vol. 8, pp. 99-111, 1992.
[41] W. Zhao, R. Chellappa, A. Rosenfeld, and P.J. Phillips, “Face Recognition: A Literature Survey,” UMD Cfar Technical Report 948, 2000.
[42] S.C. Zhu, Y.N. Wu, and D.B. Mumford, “Filters, Random Fields and Maximum Entropy (FRAME),” Int'l J. Computer Vision, vol. 27, no. 2, pp. 1-20, 1998.
[43] S.C. Zhu and D. Mumford, “Quest for a Stochastic Grammar of Images,” Foundations and Trends in Computer Graphics and Vision, 2007.
[44] Z. Xu and J. Luo, “Accurate Dynamic Sketching of Faces from Video,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, Workshop on Semantic Learning Applications in Multimedia, 2007.
18 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool