Subscribe
Issue No.04 - April (2010 vol.32)
pp: 733-746
Gabriel Peyré , Université Paris-Dauphine, CEREMADE, Paris
ABSTRACT
This paper proposes a new method to synthesize and inpaint geometric textures. The texture model is composed of a geometric layer that drives the computation of a new grouplet transform. The geometry is an orientation flow that follows the patterns of the texture to analyze or synthesize. The grouplet transform extends the original construction of Mallat [1] and is adapted to the modeling of natural textures. Each grouplet atoms is an elongated stroke located along the geometric flow. These atoms exhibit a wide range of lengths and widths, which is important to match the variety of structures present in natural images. Statistical modeling and sparsity optimization over these grouplet coefficients enable the synthesis of texture patterns along the flow. This paper explores texture inpainting and texture synthesis, which both require the joint optimization of the geometric flow and the grouplet coefficients.
INDEX TERMS
Texture, grouplets, texture synthesis, inpainting.
CITATION
Gabriel Peyré, "Texture Synthesis with Grouplets", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 4, pp. 733-746, April 2010, doi:10.1109/TPAMI.2009.54
REFERENCES
 [1] S. Mallat, "Geometrical Grouplets," Applied and Computational Harmonic Analysis, 2008. [2] J.R. Bergen and M.S. Landy, "Computational Modeling of Visual Texture Segregation," Computational Models of Visual Processing, M.S. Landy and J.A. Movshon, eds., pp. 253-271, MIT Press, 1991. [3] M. Kass and A. Witkin, "Analyzing Oriented Patterns," Computer Vision Graphics and Image Processing, vol. 37, no. 3, pp. 362-385, 1987. [4] C.F. Shu and R.C. Jain, "Vector Field Analysis for Oriented Patterns," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 9, pp. 946-950, Sept. 1994. [5] U. Kothe, "Edge and Junction Detection with an Improved Structure Tensor," Proc. DAGM '03, pp. 25-32, 2003. [6] A. Rao and R. Jain, "Computerized Flow Field Analysis: Oriented Texture Fields," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 7, pp. 693-709, July 1992. [7] M. Almeida, "Anisotropic Textures with Arbitrary Orientation," J. Math. Imaging and Vision, vol. 7, no. 3, pp. 241-251, June 1997. [8] O. Ben-Shahar and S. Zucker, "The Perceptual Organization of Texture Flow: A Contextual Inference Approach," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 4, pp. 401-417, Apr. 2003. [9] A. Jain, L. Hong, and R. Bolle, "On-Line Fingerprint Verification," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 4, pp. 302-314, Apr. 1997. [10] S. Mallat, A Wavelet Tour of Signal Processing. Academic Press, 1998. [11] E.P. Simoncelli, W.T. Freeman, E.H. Adelson, and D.J. Heeger, "Shiftable Multi-Scale Transforms," IEEE Trans. Information Theory, vol. 38, no. 2, pp. 587-607, Mar. 1992. [12] F.G. Meyer and R.R. Coifman, "Brushlets: A Tool for Directional Image Analysis and Image Compression," J. Applied and Computational Harmonic Analysis, vol. 5, pp. 147-187, 1997. [13] E. Candès and D. Donoho, "New Tight Frames of Curvelets and Optimal Representations of Objects with Piecewise ${\rm C}^2$ Singularities," Comm. Pure and Applied Math., vol. 57, no. 2, pp. 219-266, 2004. [14] L. Demanet and L. Ying, "Wave Atoms and Sparsity of Oscillatory Patterns," Applied and Computational Harmonic Analysis, vol. 23, no. 3, pp. 368-387, 2007. [15] E. Le Pennec and S. Mallat, "Bandelet Image Approximation and Compression," SIAM Multiscale Modeling and Simulation, vol. 4, no. 3, pp. 992-1039, 2005. [16] S. Mallat and G. Peyré, "Orthogonal Bandlets for Geometric Image Approximation," Comm. Pure and Applied Math., vol. 61, no. 9, pp. 1173-1212, 2008. [17] S. Masnou, "Disocclusion: A Variational Approach Using Level Lines," IEEE Trans. Image Processing, vol. 11, no. 2, pp. 68-76, Feb. 2002. [18] C. Ballester, M. Bertalmio, V. Caselles, G. Sapiro, and J. Verdera, "Filling-In by Joint Interpolation of Vector Fields and Gray Levels," IEEE Trans. Image Processing, vol. 10, no. 8, pp. 1200-1211, Aug. 2001. [19] M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, "Image Inpainting," Proc. ACM SIGGRAPH' 00, pp. 417-424, 2000. [20] D. Tschumperlé and R. Deriche, "Vector-Valued Image Regularization with PDEs: A Common Framework for Different Applications," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 506-517, Apr. 2005. [21] M. Elad, J.-L. Starck, D. Donoho, and P. Querre, "Simultaneous Cartoon and Texture Image Inpainting Using Morphological Component Analysis (MCA)," J. Applied and Computational Harmonic Analysis, vol. 19, pp. 340-358, 2005. [22] M. Fadili, J.-L. Starck, and F. Murtagh, "Inpainting and Zooming Using Sparse Representations," Computer J., 2007. [23] S. Mallat and G. Peyré, "A Review of Bandlet Methods for Geometrical Image Representation," Numerical Algorithms, vol. 44, no. 3, pp. 205-234, 2007. [24] J.-P. Lewis, "Texture Synthesis for Digital Painting," ACM SIGGRAPH Computer Graphics, vol. 18, no. 3, pp. 245-252, 1984. [25] K. Perlin, "An Image Synthesizer," Proc. ACM SIGGRAPH '85, pp. 287-296, 1985. [26] D.J. Heeger and J.R. Bergen, "Pyramid-Based Texture Analysis/Synthesis," Proc. ACM SIGGRAPH '95, R. Cook, ed., pp. 229-238, Aug. 1995. [27] R. Cook and T. DeRose, "Wavelet Noise," ACM Trans. Graphics, vol. 24, no. 3, pp. 803-811, 2005. [28] R. Paget and I.D. Longstaff, "Texture Synthesis via a Noncausal Nonparametric Multiscale Markov Random Field," IEEE Trans. Image Processing, vol. 7, no. 6, pp. 925-931, June 1998. [29] J.S.D. Bonet, "Multiresolution Sampling Procedure for Analysis and Synthesis of Texture Images," Proc. ACM SIGGRAPH '97, pp. 361-368, 1997. [30] S.C. Zhu, Y. Wu, and D. Mumford, "Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling," Int'l J. Computer Vision, vol. 27, no. 2, pp. 107-126, 1998. [31] J. Portilla and E.P. Simoncelli, "A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients," Int'l J. Computer Vision, vol. 40, no. 1, pp. 49-70, 2000. [32] A. Witkin and M. Kass, "Reaction-Diffusion Textures," Proc. ACM SIGGRAPH '91, pp. 299-308, 1991. [33] J. Weickert, "Coherence-Enhancing Diffusion Filtering," Int'l J. Computer Vision, vol. 31, nos. 2/3, pp. 111-127, 1999. [34] B. Cabral and L.C. Leedom, "Imaging Vector Fields Using Line Integral Convolution," Proc. ACM SIGGRAPH '93, pp. 263-272, 1993. [35] J.J. van Wijk, "Image Based Flow Visualization," Proc. ACM SIGGRAPH '02, pp. 745-754, 2002. [36] U. Diewald, T. Preußer, and M. Rumpf, "Anisotropic Diffusion in Vector Field Visualization on Euclidean Domains and Surfaces," IEEE Trans. Visualization and Computer Graphics, vol. 6, no. 2, pp. 139-149, Apr.-June 2000. [37] A.A. Efros and T.K. Leung, "Texture Synthesis by Non-Parametric Sampling," Proc. Int'l Conf. Computer Vision, vol. 2, p. 1033, 1999. [38] L.-Y. Wei and M. Levoy, "Fast Texture Synthesis Using Tree-Structured Vector Quantization," Proc. ACM SIGGRAPH '00, pp. 479-488, 2000. [39] M. Ashikhmin, "Synthesizing Natural Textures," Proc. Symp. Interactive 3D Graphics, pp. 217-226, 2001. [40] L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, "Real-Time Texture Synthesis by Patch-Based Sampling," ACM Trans. Graphics, vol. 20, no. 3, pp. 127-150, 2001. [41] A. Efros and W. Freeman, "Image Quilting for Texture Synthesis and Transfer," Proc. ACM SIGGRAPH '01, pp. 341-346, Aug. 2001. [42] V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, "Graphcut Textures: Image and Video Synthesis Using Graph Cuts," ACM Trans. Graphics, vol. 22, no. 3, pp. 277-286, July 2003. [43] S. Lefebvre and H. Hoppe, "Parallel Controllable Texture Synthesis," ACM Trans. Graphics, vol. 24, no. 3, pp. 777-786, 2005. [44] V. Kwatra, I. Essa, A. Bobick, and N. Kwatra, "Texture Optimization for Example-Based Synthesis," ACM Trans. Graphics, vol. 24, no. 3, pp. 795-802, July 2005. [45] S. Lefebvre and H. Hoppe, "Appearance-Space Texture Synthesis," ACM Trans. Graphics, vol. 25, no. 3, pp. 541-548, July 2006. [46] H. Chen and S.C. Zhu, "A Generative Sketch Model for Human Hair Analysis and Synthesis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 7, pp. 1025-1040, July 2006. [47] R. Cappelli, D. Maio, A. Lumini, and D. Maltoni, "Fingerprint Image Reconstruction from Standard Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1489-1503, Sept. 2007. [48] N. Foster and D.N. Metaxas, "Realistic Animation of Liquids," Graphical Models and Image Processing, vol. 58, no. 5, pp. 471-483, Sept. 1996. [49] J. Stam, "Stable Fluids," Proc. ACM SIGGRAPH '99, pp. 121-128, Aug. 1999. [50] R. Bridson, J. Houriham, and M. Nordenstam, "Curl-Noise for Procedural Fluid Flow," ACM Trans. Graphics, vol. 26, no. 3, p. 46, 2007. [51] F. Neyret, "Advected Textures," Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation, pp. 147-153, July 2003. [52] J. Stam and E. Fiume, "Depicting Fire and Other Gaseous Phenomena Using Diffusion Processes," Proc. ACM SIGGRAPH '95, pp. 129-136, 1995. [53] A. Lamorlette and N. Foster, "Structural Modeling of Natural Flames," Proc. ACM SIGGRAPH '02, S. Spencer, ed., vol. 21, pp. 729-735, July 2002. [54] T. Kim, N. Thürey, D. James, and M. Gross, "Wavelet Turbulence for Fluid Simulation," Proc. ACM SIGGRAPH '08, pp. 1-6., 2008. [55] G. Doretto, A. Chiuso, Y.N. Wu, and S. Soatto, "Dynamic Textures," Int'l J. Computer Vision, vol. 51, no. 2, pp. 91-109, Feb. 2003. [56] A.W. Bargteil, F. Sin, J.E. Michaels, T.G. Goktekin, and J.F. O'Brien, "A Texture Synthesis Method for Liquid Animations," Proc. Symp. Computer Animation, pp. 345-351, 2006. [57] V. Kwatra, D. Adalsteinsson, T. Kim, N. Kwatra, M. Carlson, and M.C. Lin, "Texturing Fluids," IEEE Trans. Visualization and Computer Graphics, vol. 13, no. 5, pp. 939-952, Sept./Oct. 2007. [58] Z. Bar-Joseph, R. El-Yaniv, D. Lischinski, and M. Werman, "Texture Mixing and Texture Movie Synthesis Using Statistical Learning," IEEE Trans. Visualization and Computer Graphics, vol. 7, no. 2, pp. 120-135, Apr.-June 2001. [59] X. Tricoche and G. Scheuermann, "Topology Simplification of Symmetric, Second-Order 2D Tensor Fields," Geometric Modeling Methods in Scientific Visualization, pp. 275-292, Springer, 2003. [60] D. Donoho and I. Johnstone, "Ideal Spatial Adaptation via Wavelet Shrinkage," Biometrika, vol. 81, pp. 425-455, Dec. 1994. [61] I. Daubechies, M. Defrise, and C.D. Mol, "An Iterative Thresholding Algorithm for Linear Inverse Problems with a Sparsity Constraint," Comm. Pure and Applied Math., vol. 57, pp. 1413-1541, 2004. [62] M. Elad, P. Milanfar, and R. Rubinstein, "Analysis Versus Synthesis in Signal Priors," Inverse Problems, vol. 23, pp. 947-968, June 2007. [63] G. Peyré, "Best Basis Compressed Sensing," Preprint Ceremade, 2007. [64] P. Brodatz, Textures: A Photographic Album for Artists and Designers. Dover Publications, 1966. [65] R.C. Gonzales and R.E. Woods, Digital Image Processing. Addison-Wesley Publishing Company, 1993. [66] G. Peyré, "Manifold Models for Signals and Images," Computer Vision and Image Understanding, 2008. [67] M. Farge and K. Schneider, "Wavelets: Application to Turbulence," Encyclopedia of Math. Physics, pp. 408-420, Elsevier, 2006.