This Article 
 Bibliographic References 
 Add to: 
Texture Mixing and Texture Movie Synthesis Using Statistical Learning
April-June 2001 (vol. 7 no. 2)
pp. 120-135

Abstract—We present an algorithm based on statistical learning for synthesizing static and time-varying textures matching the appearance of an input texture. Our algorithm is general and automatic and it works well on various types of textures, including 1D sound textures, 2D texture images, and 3D texture movies. The same method is also used to generate 2D texture mixtures that simultaneously capture the appearance of a number of different input textures. In our approach, input textures are treated as sample signals generated by a stochastic process. We first construct a tree representing a hierarchical multiscale transform of the signal using wavelets. From this tree, new random trees are generated by learning and sampling the conditional probabilities of the paths in the original tree. Transformation of these random trees back into signals results in new random textures. In the case of 2D texture synthesis, our algorithm produces results that are generally as good as or better than those produced by previously described methods in this field. For texture mixtures, our results are better and more general than those produced by earlier methods. For texture movies, we present the first algorithm that is able to automatically generate movie clips of dynamic phenomena such as waterfalls, fire flames, a school of jellyfish, a crowd of people, etc. Our results indicate that the proposed technique is effective and robust.

[1] Z. Bar-Joseph, S. Dubnov, R. El-Yaniv, D. Lischinski, and M. Werman, “Statistical Learning of Granular Synthesis Parameters with Applications for Sound Texture Synthesis,” Proc. Int'l Computer Music Conf. (ICMC99), 1999.
[2] M. Basseville, A. Benveniste, K.C. Chou, S.A. Golden, R. Nikoukhah, and A.S. Willsky, “Modeling and Estimation of Multiresolution Stochastic Processes,” IEEE Trans. Information Theory, vol. 38, no. 2, pp. 766-784, 1992.
[3] M. Basseville, A. Benveniste, and A.S. Willsky, “Multiscale Autoregressive Processes, Part II: Lattice Structures for Whitening and Modeling,” IEEE Trans. Signal Processing, vol. 40, no. 8, pp. 1935-1954, 1992.
[4] G.E.P. Box, G.M. Jenkins, G.C. Reinsel, and G. Jenkins, Time Series Analysis: Forecasting and Control. Prentice Hall, 1994.
[5] P.J. Burt and E.H. Adelson, “A Multiresolution Spline with Application to Image Mosaics,” ACM Trans. Graphics, vol. 2, no. 4, pp. 217-236, Oct. 1983.
[6] T.M. Cover and J.A. Thomas, Elements of Information Theory. John Wiley&Sons, 1991.
[7] I. Daubechies, “Orthonormal Bases of Compactly Supported Wavelets,” Comm. Pure and Applied Math., vol. 41, no. 7, pp. 909-996, Oct. 1988.
[8] J.S. De Bonet, “Multiresolution Sampling Procedure for Analysis and Synthesis of Texture Images,” SIGGRAPH '97 Conf. Proc., pp. 361-368, 1997.
[9] J.S. De Bonet, “Novel Statistical Multiresolution Techniques for Image Synthesis, Discrimination, and Recognition,” master's thesis, Massachusetts Inst. of Technology, Cambridge, Mass., May 1997.
[10] J.S. De Bonet and P. Viola, “A Non-Parametric Multi-Scale Statistical Model for Natural Images,” Advances in Neural Information Processing, vol. 10, 1997.
[11] J.S. De Bonet and P. Viola, “Texture Recognition Using a Non-Parametric Multi-Scale Statistical Model,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1998.
[12] R.A. DeVore, B. Jawerth, and B.J. Lucier, “Image Compression through Wavelet Transform Coding,” IEEE Trans. Information Theory, vol. 38, no. 2 (Part II)), pp. 719-746, 1992.
[13] R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis. John Wiley&Sons, 1973.
[14] D. Ebert, W. Carlson, and R. Parent, “Solid Spaces and Inverse Particle Systems for Controlling the Animation of Gases and Fluids,” The Visual Computer, vol. 10, no. 4, pp. 179-190, 1994.
[15] D. Ebert, K. Musgrave, D. Peachey, K. Perlin, and S. Worley, Texturing and Modeling: A Procedural Approach. Academic Press, Oct. 1994.
[16] D.S. Ebert and R.E. Parent, “Rendering and Animation of Gaseous Phenomena by Combining Fast Volume and Scanline A-Buffer Techniques,” Computer Graphics (SIGGRAPH '90 Proc.), F. Baskett, ed., vol. 24, no. 4, pp. 357-366, Aug. 1990.
[17] A. Efros and T. Leung, “Texture Synthesis by Non-Parametric Sampling,” Proc. Seventh Int'l Conf. Computer Vision, 1999.
[18] R. El-Yaniv, S. Fine, and N. Tishby, “Agnostic Classification of Markovian Sequences,” Advances in Neural Information Processing Systems, M.I. Jordan, M.J. Kearns, and S.A. Solla, eds., volume 10, MIT Press, 1998.
[19] A. Fournier and W.T. Reeves, “A Simple Model of Ocean Waves,” Computer Graphics (SIGGRAPH '86 Proc.), D.C. Evans and R.J. Athay, eds., vol. 20, no. 4, pp. 75-84, Aug. 1986.
[20] D.J. Heeger and J.R. Bergen, “Pyramid-Based Texture Analysis/Synthesis,” SIGGRAPH 95 Conf. Proc., R.L. Cook, ed., pp. 229-238, Aug. 1995.
[21] A. Karasaridis and E. Simoncelli, “A Filter Design Technique for Steerable Pyramid Image Transforms,” Proc. Int'l Conf. Acoustics, Speech, and Signal Processing (ICASSP-96), May 1996
[22] N. Merhav and M. Feder, “Universal Prediction,” IEEE Trans. Information Theory, vol. 44, no. 6, pp. 2124-2147, 1998.
[23] D.R. Peachey, “Modeling Waves and Surf,” Computer Graphics (SIGGRAPH '86 Proc.), D.C. Evans and R.J. Athay, eds., vol. 20, no. 4, pp. 65-74, Aug. 1986.
[24] K. Perlin, “An Image Synthesizer,” Computer Graphics (SIGGRAPH '85 Proc.), B.A. Barsky, ed., vol. 19, no. 3, pp. 287-296, July 1985.
[25] W.T. Reeves, “Particle Systems—A Technique for Modeling a Class of Fuzzy Objects,” ACM Trans. Graphics, vol. 2, no. 2, pp. 91-108, Apr. 1983.
[26] W.T. Reeves and R. Blau, “Approximate and Probabilistic Algorithms for Shading and Rendering Structured Particle Systems,” Computer Graphics (SIGGRAPH '85 Proc.), B.A. Barsky, ed., vol. 19, no. 3, pp. 313-322, July 1985.
[27] A. Schödl, R. Szeliski, D.H. Salesin, and I. Essa, “Video Textures,” Proc. SIGGRAPH 2000, pp. 489-498, July 2000.
[28] E.P. Simoncelli, W.T. Freeman, E.H. Adelson, and D.J. Heeger, “Shiftable Multi-Scale Transforms,” IEEE Trans. Information Theory, vol. 38, no. 2, pp. 587-607, Mar. 1992.
[29] K. Sims, “Particle Animation and Rendering Using Data Parallel Computation,” Computer Graphics (SIGGRAPH '90 Proc.), F. Baskett, ed., vol. 24, no. 4, pp. 405-413, Aug. 1990.
[30] J. Stam and E. Fiume, “Turbulent Wind Fields for Gaseous Phenomena,” Computer Graphics (SIGGRAPH '93 Proc.), J.T. Kajiya, ed., vol. 27, pp. 369-376, Aug. 1993.
[31] J. Stam and E. Fiume, “Depicting Fire and Other Gaseous Phenomena Using Diffusion Processes,” SIGGRAPH 95 Conf. Proc., R. Cook, ed., pp. 129-136, Aug. 1995.
[32] E.J. Stollnitz, T.D. DeRose, and D.H. Salesin, Wavelets for Computer Graphics: Theory and Applications. Morgan Kaufmann, 1996.
[33] M. Szummer, “Temporal Texture Modeling,” master's thesis, Massachusetts Inst. of Technology, Cambridge, May 1995, also appeared as MIT Media Lab Perceptual Computing TR #346.
[34] M. Szummer and R.W. Picard, “Temporal Texture Modeling,” Proc. IEEE Int'l Conf. Image Processing (ICIP 1996), 1996, also appeared as MIT Media Lab Perceptual Computing TR #381.
[35] G. Turk, “Generating Textures for Arbitrary Surfaces Using Reaction-Diffusion,” Computer Graphics (SIGGRAPH '91 Proc.), T.W. Sederberg, ed., vol. 25, no. 4, pp. 289-298, July 1991.
[36] V.N. Vapnik, Statistical Learning Theory, John Wiley&Sons, 1998.
[37] L.-Y. Wei and M. Levoy, “Fast Texture Synthesis Using Tree-Structured Vector Quantization,” Proc. SIGGRAPH 2000, pp. 479-488, July 2000.
[38] A. Witkin and M. Kass, “Reaction-Diffusion Textures,” Computer Graphics (SIGGRAPH '91 Proc.), T.W. Sederberg, ed., vol. 25, no. 4, pp. 299-308, July 1991.
[39] S.P. Worley, “A Cellular Texture Basis Function,” SIGGRAPH 96 Conf. Proc., H. Rushmeier, ed., pp. 291-294, Aug. 1996.
[40] G.W. Wornell and A.V. Oppenheim, “Wavelet-Based Representations for a Class of Self-Similar Signals with Application to Fractal Modulation,” IEEE Trans. Information Theory, vol. 38, no. 2, pp. 785-800, 1992.
[41] S.C. Zhu, Y. Wu, and D. Mumford, “Filters Random Fields and Maximum Entropy(Frame)—Towards a Unified Theory for Texture Modeling,” Int'l J. Computer Vision, vol. 27, no. 2, pp. 107-126, 1998.

Index Terms:
Sound textures, statistical learning, steerable filters, time-varying textures, texture mixing, texture movies, texture synthesis, wavelets.
Ziv Bar-Joseph, Ran El-Yaniv, Dani Lischinski, Michael Werman, "Texture Mixing and Texture Movie Synthesis Using Statistical Learning," IEEE Transactions on Visualization and Computer Graphics, vol. 7, no. 2, pp. 120-135, April-June 2001, doi:10.1109/2945.928165
Usage of this product signifies your acceptance of the Terms of Use.