This Article 
 Bibliographic References 
 Add to: 
Efficient Example-Based Painting and Synthesis of 2D Directional Texture
May/June 2004 (vol. 10 no. 3)
pp. 266-277

Abstract—We present a new method for converting a photo or image to a synthesized painting following the painting style of an example painting. Treating painting styles of brush strokes as sample textures, we reduce the problem of learning an example painting to a texture synthesis problem. The proposed method uses a hierarchical patch-based approach to the synthesis of directional textures. The key features of our method are: 1) Painting styles are represented as one or more blocks of sample textures selected by the user from the example painting; 2) image segmentation and brush stroke directions defined by the medial axis are used to better represent and communicate shapes and objects present in the synthesized painting; 3) image masks and a hierarchy of texture patches are used to efficiently synthesize high-quality directional textures. The synthesis process is further accelerated through texture direction quantization and the use of Gaussian pyramids. Our method has the following advantages: First, the synthesized stroke textures can follow a direction field determined by the shapes of regions to be painted. Second, the method is very efficient; the generation time of a synthesized painting ranges from a few seconds to about one minute, rather than hours, as required by other existing methods, on a commodity PC. Furthermore, the technique presented here provides a new and efficient solution to the problem of synthesizing a 2D directional texture. We use a number of test examples to demonstrate the efficiency of the proposed method and the high quality of results produced by the method.

[1] M. Ashikhmin, Synthesizing Natural Textures Proc. 2001 Symp. Interactive 3D Graphics, J.F. Hughes and C.H. Sequin, eds., pp. 217-226, 2001.
[2] J.S.D. Bonet, Multiresolution Sampling Procedure for Analysis and Synthesis of Texture Images Proc. SIGGRAPH 1997, pp. 361-368, 1997.
[3] P.J. Burt and E.H. Adelson, “The Laplacian Pyramid as a Compact Image Code,” IEEE Trans. Comm., vol. 31, no. 4, pp. 532-540, 1983.
[4] D. Comaniciu and P. Meer, Mean Shift: A Robust Approach towards Feature Space A Analysis IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[5] C.J. Curtis, S.E. Anderson, J.E. Seims, K.W. Fleischer, and D.H. Salesin, Computer-Generated Watercolor Proc. SIGGRAPH 1997, pp. 421-430, 1997.
[6] P.E. Danielsson, Euclidean Distance Mapping Computer Graphics and Image Processing, vol. 14, pp. 227-248, 1980.
[7] A. Efros and W.T. Freeman, Image Quilting for Texture Synthesis and Transfer Proc. SIGGRAPH 2001, pp. 341-346, 2001.
[8] A. Efros and T. Leung, “Texture Synthesis by Non-Parametric Sampling,” Proc. Seventh Int'l Conf. Computer Vision, 1999.
[9] W.T. Freeman, J.B. Tenenbaum, and E.C. Pasztor, An Example-Based Approach to Style Translation for Line Drawings ACM Trans. Graphics, vol. 22, no. 1, pp. 33-46, 2003.
[10] B. Gooch and A. Gooch, Non-Photorealistic Rendering. A.K. Peters Ltd., 2001.
[11] B. Gooch, G. Coombe, and P. Shirley, Artistic Vision: Painterly Rendering Using Computer Vision Techniques Proc. Int'l Symp. Non-Photorealistic Animation and Rendering (NPAR 2002), 2002.
[12] P. Haeberli, Paint by Numbers: Abstract Image Representations Proc. SIGGRAPH 1990, pp. 207-214, 1990.
[13] J. Hamel and T. Strothotte, Capturing and Re-Using Rendition Styles for Non-Photorealsitic Rendering Computer Graphics Forum (Eurographics '99), vol. 18, no. 3, pp. 173-182, 1999.
[14] P. Harrison, A Non-Hierarchical Procedure for Re-Synthesis of Complex Textures Proc. Int'l Conf. in Central Europe Computer Graphics and Visualization (WSCG 2001) pp. 190-197, 2001.
[15] A. Hausner, Simulating Decorative Mosaics Proc. SIGGRAPH 2001, pp. 573-580, 2001.
[16] D.J. Heeger and J.R. Bergen, Pyramid Based Texture Analysis/Synthesis Proc. SIGGRAPH 1995, pp. 229-238, 1995.
[17] A. Hertzmann, Painterly Rendering with Curved Brush Proc. SIGGRAPH 1998, pp. 453-460, 1998.
[18] A. Hertzmann, C.E. Jacobs, N. Oliver, B. Curless, and D.H. Salesin, Image Analogies Proc. SIGGRAPH 2001, pp. 327-340, 2001.
[19] A. Hertzmann, N. Oliver, B. Curless, and S.M. Seitz, Curve Analogies Proc. 13th Eurographics Workshop Rendering, 2002.
[20] R. Jain, R. Kasturi, and B. Schunck, Machine Vision. McGraw-Hill, 1995.
[21] A.K. Jain, Fundamentals of Digital Image Processing. Prentice-Hall, 1989.
[22] P.M. Jodoin, E. Epstein, M. Granger-Piche, and V. Ostromoukhov, Hatching by Example: A Statistical Approach Proc. Int'l Symp. Non-Photorealistic Animation and Rendering (NPAR 2002), 2002.
[23] R.D. Kalnins et al., WYSIWYG NPR: Drawing Strokes Directly on 3D Models ACM Trans. Graphics (Siggraph 2002), vol. 21, no. 3, pp. 755-762, 2002.
[24] J.-G. Leu, Computing a Shape's Moments from Its Boundary Pattern Recognition, vol. 24, no. 10, pp. 949-957, 1991.
[25] L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, Real-Time Texture Synthesis by Patch-Based Sampling ACM Trans. Graphics, vol. 20, no. 3, pp. 127-150, 2001.
[26] F. Petrie and J. Shaw, The Big Book of Painting Nature in Watercolor. Watson-Guptill, 1990.
[27] E. Praun, A. Finkelstein, and H. Hoppe, Lapped Textures Proc. SIGGRAPH 2000, pp. 465-470, 2000.
[28] J.C. Russ, The Image Processing Handbook. CRC Press, 1995.
[29] M.P. Salisbury, M.T. Wong, J.F. Hughes, and D.H. Salesin, Orientable Textures for Image-Based Pen-and-Ink Illustration Proc. SIGGRAPH 1997, pp. 401-406, 1997.
[30] S.A. Schaeffer and J. Shaw, The Big Book of Painting Nature in Pastel. Watson-Guptill, 1993.
[31] M. Shiraishi and Y. Yamaguchi, An Algorithm for Automatic Painterly Rendering Based on Local Source Image Approximation Proc. Int'l Symp. Non-Photorealistic Animation and Rendering (NPAR 2000), pp. 53-58, 2000.
[32] C. Soler, M.-P. Cani, and A. Angelidis, Hierarchical Pattern Mapping ACM Trans. Graphics (Siggraph 2002), vol. 21, no. 3, pp. 673-680, 2002.
[33] S. Strassmann, Hairy Brushes Proc. SIGGRAPH 1986, pp. 225-232, 1986.
[34] H. Tamura, S. Mori, and T. Yamawaki, Textural Features Corresponding to Visual Perception IEEE Trans. Systems, Man, and Cybernetics, vol. 8, pp. 460-472, 1978.
[35] X. Tong, J. Zhang, L. Liu, X. Wang, B. Guo, and H.-Y. Shum, Synthesis of Bidirectional Texture Functions on Arbitrary Surfaces ACM Trans. Graphics (Siggraph 2002), vol. 21, no. 3, pp. 665-672, 2002.
[36] G. Turk and D. Banks, Image-Guided Streamline Placement Proc. SIGGRAPH 1996, pp. 453-460, 1996.
[37] G. Turk, Texture Synthesis on Surfaces Proc. SIGGRAPH 2001, pp. 347-354, 2001.
[38] L.-Y. Wei and M. Levoy, Fast Texture Synthesis Using Tree-Structured Vector Quantization Proc. SIGGRAPH 2000, pp. 479-488, 2000.
[39] L.-Y. Wei and M. Levoy, Texture Synthesis over Arbitrary Manifold Surfaces Proc. SIGGRAPH 2001, pp. 355-360, 2001.
[40] Y.-Q. Xu, B. Guo, and H.-Y. Shum, Chaos Mosaic: Fast and Memory Efficient Texture Synthesis Technical Report 32, Microsoft Research Asia, 2000.
[41] L.-X. Ying, A. Hertzmann, H. Biermann, and D. Zorin, Texture and Shape Synthesis on Surfaces Proc. 12th Eurographics Workshop Rendering, pp. 301-312, 2001.

Index Terms:
Digital painting, example-based painting, painting style, artistic filter, painting systems, simulation, image segmentation, Gaussian pyramid, texture synthesis, directional texture, nonphotorealistic rendering.
Bin Wang, Wenping Wang, Huaiping Yang, Jiaguang Sun, "Efficient Example-Based Painting and Synthesis of 2D Directional Texture," IEEE Transactions on Visualization and Computer Graphics, vol. 10, no. 3, pp. 266-277, May-June 2004, doi:10.1109/TVCG.2004.1272726
Usage of this product signifies your acceptance of the Terms of Use.