This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Inference of Segmented Color and Texture Description by Tensor Voting
June 2004 (vol. 26 no. 6)
pp. 771-786
Jiaya Jia, IEEE
Chi-Keung Tang, IEEE Computer Society

Abstract—A robust synthesis method is proposed to automatically infer missing color and texture information from a damaged 2D image by N\rm D tensor voting (N > 3). The same approach is generalized to range and 3D data in the presence of occlusion, missing data and noise. Our method translates texture information into an adaptive N\rm D tensor, followed by a voting process that infers noniteratively the optimal color values in the N\rm D texture space. A two-step method is proposed. First, we perform segmentation based on insufficient geometry, color, and texture information in the input, and extrapolate partitioning boundaries by either 2D or 3D tensor voting to generate a complete segmentation for the input. Missing colors are synthesized using N\rm D tensor voting in each segment. Different feature scales in the input are automatically adapted by our tensor scale analysis. Results on a variety of difficult inputs demonstrate the effectiveness of our tensor voting approach.

[1] C. Ballester, V. Caselles, J. Verdera, M. Bertalmio, and G. Sapiro, A Variational Model for Filling-In Proc. Int'l Conf. Image Processing, 2003.
[2] M. Bertalmio, G. Sapiro, C. Ballester, and V. Caselles, Image Inpainting Proc. SIGGRAPH 2000 Conf., pp. 417-424, 2000.
[3] M. Bertalmio, L. Vese, G. Sapiro, and S. Osher, Simultaneous Structure and Texture Image Inpainting Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 707-712, 2003.
[4] J. Bigün, Local Symmetry Features in Image Processing PhD thesis, Linköping Univ., Sweden, 1988.
[5] J.S. De Bonet, Multiresolution Sampling Procedure for Analysis and Synthesis of Texture Images Proc. SIGGRAPH'97 Conf., pp. 361-368, 1997.
[6] R.A. Brooks, Model-Based Three-Dimensional Interpretations of Two-Dimensional Images IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 5, no. 2, pp. 140-150, 1983.
[7] J.C. Carr, W.R. Fright, and R.K. Beatson, Surface Interpolation with Radial Basis Functions for Medical Imaging IEEE Trans. Medical Imaging, vol. 16, no. 1, pp. 96-107, 1997.
[8] T. Chan and J. Shen, Non-Texture Inpaintings by Curvature-Driven Diffusions Technical Report 00-35, Dept. of Math., Univ. of Calif. at Los Angeles, 2000.
[9] C.-F. Chang, G. Bishop, and A. Lastra, Ldi Tree: A Hierarchical Representation for Image-Based Rendering Proc. 26th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 291-298, 1999.
[10] A. Criminisi, P. Perez, and K. Toyama, Object Removal by Exemplar-Based Inpainting Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 721-728, 2003.
[11] B. Curless and M. Levoy, A Volumetric Method for Building Complex Models from Range Images Proc. 23rd Ann. Conf. Computer Graphics and Interactive Techniques, pp. 303-312, 1996.
[12] J. Davis, S. Marschner, M. Garr, and M. Levoy, Filling Holes in Complex Surfaces Using Volumetric Diffusion Proc. First Int'l Symp. 3D Data Processing, Visualization, and Transmission, 2002.
[13] Y. Deng and B. S. Manjunath, Unsupervised Segmentation of Color-Texture Regions in Images and Video IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 8, pp. 800-810, Aug. 2001.
[14] I. Drori, D. Cohen-Or, and H. Yeshurun, Fragment-Based Image Completion ACM Trans. Graphics, vol. 22, no. 3, pp. 303-312, 2003.
[15] A. Efros and T. Leung, “Texture Synthesis by Non-Parametric Sampling,” Proc. Seventh Int'l Conf. Computer Vision, 1999.
[16] A.A. Efros and W.T. Freeman, Image Quilting for Texture Synthesis and Transfer Proc. 28th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 341-346, 2001.
[17] I. Guskov and Z. Wood, Topological Noise Removal Proc. Graphics Interface Conf.,, 2001.
[18] D.J. Heeger and J.R. Bergen, Pyramid-Based Texture Analysis/Synthesis Proc. 22nd Ann. Conf. Computer Graphics and Interactive Techniques, pp. 229-238, 1995.
[19] H. Hoppe, T. DeRose, T. Duchamp, J. McDonald, and W. Stuetzle, Surface Reconstruction from Unorganized Points Proc. 19th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 71-78, 1992.
[20] H. Hoppe, T. DeRose, T. Duchamp, J. McDonald, and W. Stuetzle, Mesh Optimization Proc. 20th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 19-26, 1993.
[21] J. Jia and C.-K. Tang, Image Repairing: Robust Image Synthesis by Adaptive ND Tensor Voting Proc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 643-650, 2003.
[22] S. Kang, R. Szeliski, and J. Chai, Handling Occlusions in Dense Multi-View Stereo Proc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 103-110, 2001.
[23] M. Kass, A. Witkin, and D. Terzopoulous, Snakes: An Active Contour Models Int'l J. Computer Vision, vol. 1, no. 4, pp. 321-331, 1987.
[24] A. Levin, A. Zomet, and Y. Weiss, Learning How to Inpaint from Global Image Statistics Proc. Int'l Conf. Computer Vision, pp. 305-312, 2003.
[25] Y. Li, C.-K. Tang, and H.-Y. Shum, Efficient Dense Depth Estimation from Dense Multiperspective Panoramas Proc. Eighth Int'l Conf. Computer Vision, pp. 119-126, 2001.
[26] P. Liepa, Filling Holes in Meshes Proc. Eurographics Symp. Geometry Processing, vol. 2, pp. 721-728, 2003.
[27] W.R. Mark, L. McMillan, and G. Bishop, Post-Rendering 3D Warping Proc. 1997 Symp. Interactive 3D Graphics, pp. 7-16, 1997.
[28] S. Masnou and J. Morel, Level Line(s) Based Disocclusion Proc. Int'l Conf. Image Processing, pp. 259-263, 1998.
[29] G. Medioni, M.S. Lee, and C.K. Tang, A Computational Framework for Feature Extraction and Segmentation. Elseviers Science, 2000.
[30] E.N. Mortensen and W.A. Barrett, Intelligent Scissors for Image Composition Proc. 22nd Ann. Conf. Computer Graphics and Interactive Techniques, pp. 191-198, 1995.
[31] D.R. Peachey, Solid Texturing of Complex Surfaces Proc. 12th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 279-286, 1985.
[32] P. Pérez, A. Blake, and M. Gangnet, Jetstream: Probabilistic Contour Extraction with Particles Proc. Int'l Conf. Computer Vision, July 2001.
[33] K. Perlin, An Image Synthesizer Proc. 12th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 287-296, 1985.
[34] R. Pfeifle and H.-P. Seidel, Triangular B-Splines for Blending and Filling of Polygonal Holes Proc. Graphics Interface Conf., pp. 186-193, 1996.
[35] J. Portilla and E.P. Simoncelli, A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients Int'l J. Computer Vision, vol. 40, no. 1, pp. 49-70, Oct. 2000.
[36] J. Gårding and T. Lindeberg, Direct Computation of Shape Cues Using Scale-Adapted Spatial Derivative Operators Int'l J. Computer Vision, vol. 17, no. 2, pp. 163-191, 1996.
[37] H.-Y. Shum and L.-W. He, Rendering with Concentric Mosaics Proc. 26th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 299-306, 1999.
[38] J.M. Snyder and J.T. Kajiya, Generative Modeling: A Symbolic System for Geometric Modeling Proc. 19th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 369-378, 1992.
[39] W.-S. Tong and C.-K. Tang, Multiscale Surface Reconstruction by Tensor Voting technical report, Dept. of Computer Science, Hong Kong Univ. of Science and Tech nology, 2004.
[40] G. Turk and J.F. O'Brien, Variational Implicit Surfaces Technical Report GIT-GVU-99-15, Graphics, Visualization, and Useability Center, Georgia Inst. of Tech nology, 1999.
[41] J. Verdera, V. Caselles, M. Bertalmio, and G. Sapiro, Inpainting Surface Holes Proc. Int'l Conf. Image Processing, 2003.
[42] J. Wang and M.M. Oliveira, A Hole Filling Strategy for Reconstruction of Smooth Surfaces in Range Images Proc. Brazilian Symp. Computer Graphics and Image Processing (SIBGRAPI03), 2003.
[43] L.-Y. Wei and M. Levoy, Fast Texture Synthesis Using Tree-Structured Vector Quantization Proc. 27th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 479-488, 2000.
[44] A. Witkin and M. Kass, Reaction-Diffusion Textures Proc. 18th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 299-308, 1991.
[45] Y.Q. Xu, S.C. Zhu, B.N. Guo, and H.Y. Shum, Asymptotically Admissible Texture Synthesis Proc. Second Int'l Workshop Statistical and Computational Theories of Vision, 2001.
[46] H. Yamauchi, J. Haber, and H.-P. Seidel, Image Restoration Using Multiresolution Texture Synthesis and Image Inpainting Proc. Computer Graphics International (CGI) Conf., pp. 120-125, 2003.

Index Terms:
Image restoration, segmentation, color, texture, tensor voting, applications.
Citation:
Jiaya Jia, Chi-Keung Tang, "Inference of Segmented Color and Texture Description by Tensor Voting," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 6, pp. 771-786, June 2004, doi:10.1109/TPAMI.2004.10
Usage of this product signifies your acceptance of the Terms of Use.