The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - March/April (2008 vol.14)
pp: 426-439
ABSTRACT
This paper presents an approach of replacing textures of specified regions in the input image/video with new ones. The replacement results have the similar distortion and shading effects conforming to the underlying geometry and lighting conditions. For replacing textures in single image, the approach consists of two steps. First, a stretchbased mesh parametrization incorporating the recovered normal information is deduced to imitate perspective distortion of the interest region. Second, a Poisson-based refinement process is exploited to account for texture distortion at fine scale. Our approach is independent of the replaced textures. Once processing the input image is completed, any new texture can be applied efficiently. For dealing with video sequence, one key-frame based texture replacement approach is devised. The approach is generalized from image retexturing. It repeatedly propagates the replacement results of key frames to the rest ones. We develop a local motion optimization scheme to deal with the inaccuracies of optical flow when tracking moving objects. One graphcut segmentation algorithm is incorporated into the approach for handling visibility shifting. Texture drifting is alleviated with one globally optimization to smooth trajectories of the tracked points over temporal domain. Experimental results show that our approach can generate visually pleasing results for both image and video.
INDEX TERMS
Picture/Image Generation, Color, shading, shadowing, and texture, Image Processing and Computer Vision
CITATION
Hanqiu Sun, Yanwen Guo, Zhongding Jiang, "Mesh-Guided Optimized Retexturing for Image and Video", IEEE Transactions on Visualization & Computer Graphics, vol.14, no. 2, pp. 426-439, March/April 2008, doi:10.1109/TVCG.2007.70438
REFERENCES
[1] H. Fang and J. Hart, “Textureshop: Texture Synthesis as a Photograph Editing Tool,” Proc. ACM SIGGRAPH '04/ACM Trans. Graphics, vol. 23, no. 3, pp. 354-359, 2004.
[2] Y. Liu, W.-C. Lin, and J. Hays, “Near Regular Texture Analysis and Manipulation,” Proc. ACM SIGGRAPH '04/ACM Trans. Graphics, vol. 23, no. 3, pp. 368-376, 2004.
[3] Y. Tsin, Y. Liu, and V. Ramesh, “Texture Replacement in Real Images,” Proc. IEEE Computer Vision and Pattern Recognition (CVPR '01), vol. 2, pp. 539-544, 2001.
[4] H. Fang and J. Hart, “Rototexture: Automated Tools for Texturing Raw Video,” IEEE Trans. Visualization and Computer Graphics, vol. 12, no. 6, pp. 1580-1589, Nov./Dec. 2006.
[5] V. Scholz and M. Magnor, “Texture Replacement of Garments in Monocular Video Sequences,” Proc. 17th Eurographics Symp. Rendering, pp. 305-312, 2006.
[6] B. Oh, M. Chen, J. Dorsey, and F. Durand, “Image-Based Modeling and Photo Editing,” Proc. ACM SIGGRAPH '01, pp.433-442, 2001.
[7] B. K. Horn, “Height and Gradient from Shading,” Int'l J. Computer Vision, vol. 5, no. 1, pp. 37-75, 1990.
[8] S. Zelinka, H. Fang, M. Garland, and J. Hart, “Interactive Material Replacement in Photographs,” Proc. Graphics Interface, pp. 227-232, 2005.
[9] Y. Li, J. Sun, C.-K. Tang, and H.-Y. Shum, “Lazy Snapping,” Proc. ACM SIGGRAPH '04/ACM Trans. Graphics, vol. 23, no. 3, pp. 303-308, 2004.
[10] S. Zelinka and M. Garland, “Towards Real-Time Texture Synthesis with the Jump Map,” Proc. 13th Eurographics Workshop Rendering Techniques, pp. 99-104, 2002.
[11] E.A. Khan, E. Reinhard, R.W. Fleming, and H.H. Bülthoff, “Image-Based Material Editing,” Proc. ACM SIGGRAPH '06/ACM Trans. Graphics, vol. 25, no. 3, pp. 654-663, 2006.
[12] R. White and D. Forsyth, “Retexturing Single Views Using Texture and Shading,” Proc. European Conf. Computer Vision (ECCV '06), vol. 3, pp. 225-239, 2006.
[13] S.S. Beauchemin and J.L. Barron, “The Computation of Optical Flow,” ACM Computing Surveys, vol. 27, no. 3, pp. 433-467, 1995.
[14] J. Shi and C. Tomasi, “Good Features to Track,” Proc. IEEE Computer Vision and Pattern Recognition (CVPR '94), pp. 593-600, 1994.
[15] H. Jin, P. Favaro, and S. Soatto, “Real-Time Feature Tracking and Outlier Rejection with Changes in Illumination,” Proc. IEEE Int'l Conf. Computer Vision (ICCV '01), vol. 1, pp. 684-689, 2001.
[16] A. Agarwala, A. Hertzmann, D.H. Salesin, and S.M. Seitz, “Keyframe-Based Tracking for Rotoscoping and Animation,” Proc. ACM SIGGRAPH '04/ACM Trans. Graphics, vol. 23, no. 3, pp. 584-591, 2004.
[17] J. Wang, Y. Xu, H.-Y. Shum, and M. Cohen, “Video Tooning,” Proc. ACM SIGGRAPH '04/ACM Trans. Graphics, vol. 23, no. 3, pp.574-583, 2004.
[18] Y.-Y. Chuang, A. Agarwala, B. Curless, D. Salesin, and R. Szeliski, “Video Matting of Complex Scenes,” Proc. ACM SIGGRAPH '02/ACM Trans. Graphics, vol. 21, no. 3, pp. 243-248, 2002.
[19] M.J. Black, D.J. Fleet, and Y. Yacoob, “Robustly Estimating Changes in Image Appearance,” Computer Vision and Image Understanding, vol. 78, no. 1, pp. 8-31, 2000.
[20] Y. Altunbasak and A.M. Tekalp, “Closed-Form Connectivity-Preserving Solutions for Motion Compensation Using 2-D Meshes,” IEEE Trans. Image Processing, vol. 6, no. 9, pp. 1255-1269, 1997.
[21] C. Rother, V. Kolmogorov, and A. Blake, “Grabcut Interactive Foreground Extraction Using Iterated Graph Cuts,” Proc. ACM SIGGRAPH '04/ACM Trans. Graphics, vol. 23, no. 3, pp. 309-314, 2004.
[22] J. Maillot, H. Yahia, and A. Verroust, “Interactive Texture Mapping,” Proc. ACM SIGGRAPH '93, pp. 27-34, 1993.
[23] M. Eck, T. DeRose, T. Duchamp, H. Hoppe, M. Lounsbery, and W. Stuetzle, “Mutiresolution Analysis of Arbitrary Meshes,” Proc. ACM SIGGRAPH '95, pp. 173-182, 1995.
[24] S. Haker, S. Angenent, A. Tannenbaum, R. Kikinis, G. Sapiro, and M. Halle, “Conformal Surface Parametrization for Texture Mapping,” IEEE Trans. Visualization and Computer Graphics, vol. 6, no. 2, pp. 181-189, Apr.-June 2000.
[25] P.V. Sander, J. Snyder, S.J. Gortler, and H. Hoppe, “Texture Mapping Progressive Meshes,” Proc. ACM SIGGRAPH '01, pp.409-416, 2001.
[26] B. Levy, S. Petiejean, N. Ray, and J. Maillot, “Least Squares Conformal Maps for Automatic Texture Atlas Generation,” Proc. ACM SIGGRAPH '02/ACM Trans. Graphics, vol. 21, no. 3, pp. 362-371, 2002.
[27] A. Sheffer and E. de Sturler, “Smoothing an Overlay Grid to Minimize Linear Distortion in Texture Mapping,” ACM Trans. Graphics, vol. 21, no. 4, pp. 874-890, 2002.
[28] C. Gotsman, X. Gu, and A. Sheffer, “Fundamentals of Spherical Parameterization for 3D Meshes,” Proc. ACM SIGGRAPH '03/ACM Trans. Graphics, vol. 22, no. 3, pp. 358-363, 2003.
[29] K. Zhou, J. Snyder, B. Guo, and H.-Y. Shum, “Iso-Charts: Stretch-Driven Mesh Parameterization Using Spectral Analysis,” Proc. ACM Symp. Geometry Processing, pp. 45-54, 2004.
[30] http:/mathworld.wolfram.com/, 2007.
[31] J. Tohline, Origin of the Poisson Equation, http://www.phys.lsu. edu/astro/HBook.current/ Context/PGEpoisson.origin.text.pdf , 1999.
[32] P. Perez, M. Gangnet, and A. Blake, “Poisson Image Editing,” Proc. ACM SIGGRAPH '03, vol. 22, no. 3, pp. 313-318, 2003.
[33] A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen, “Interactive Digital Photomontage,” Proc. ACM SIGGRAPH '04/ACM Trans. Graphics, vol. 23, no. 3, pp. 294-302, 2004.
[34] Y. Yu, K. Zhou, D. Xu, X. Shi, H. Bao, B. Guo, and H.-Y. Shum, “Mesh Editing with Poisson-Based Gradient Field Manipulation,” Proc. ACM SIGGRAPH '04/ACM Trans. Graphics, vol. 23, no. 3, pp.641-648, 2004.
[35] D. Xu, H. Zhang, Q. Wang, and H. Bao, “Poisson Shape Interpolation,” Proc. ACM Symp. Solid and Physical Modeling, pp.267-274, 2005.
[36] Z. Jiang, T.-T. Wong, and H. Bao, “Practical Super-Resolution from Dynamic Video Sequences,” Proc. IEEE Computer Vision and Pattern Recognition (CVPR '03), vol. 2, pp. 549-554, 2003.
[37] D. Shepard, “A Two Dimensional Interpolation Function for Irregularly Spaced Data,” Proc. 23rd Nat'l Conf. ACM, pp. 517-524, 1968.
[38] C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” Proc. IEEE Computer Vision and Pattern Recognition (CVPR '98), pp. 839-846, 1998.
[39] S. Fleishman, I. Drori, and D. Cohen-Or, “Bilateral Mesh Denoising,” Proc. ACM SIGGRAPH '03/ACM Trans. Graphics, vol. 22, no. 3, pp. 950-953, 2003.
[40] Y. Boykov and M.P. Jolly, “Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in n-d Images,” Proc. IEEE Int'l Conf. Computer Vision (ICCV '01), vol. 1, pp. 105-112, 2001.
[41] Right Hemisphere, http://www.righthemisphere.com/productsdexp /, 2007.
[42] J. Zhang, K. Zhou, L. Velho, B. Guo, and H.-Y. Shum, “Synthesis of Progressively-Variant Texture on Arbitrary Surfaces,” Proc. ACM SIGGRAPH '03/ACM Trans. Graphics, vol. 22, no. 3, pp. 295-302, 2003.
[43] Y. Li, J. Sun, and H.-Y. Shum, “Video Object Cut and Paste,” Proc. ACM SIGGRAPH '05/ACM Trans. Graphics, vol. 24, pp. 595-600, 2005.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool