The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - November (2009 vol.31)
pp: 2032-2047
Manik Varma , Microsoft Research, Bangalore
Andrew Zisserman , University of Oxford, Oxford
ABSTRACT
In this paper, we investigate material classification from single images obtained under unknown viewpoint and illumination. It is demonstrated that materials can be classified using the joint distribution of intensity values over extremely compact neighborhoods (starting from as small as 3 \times 3 pixels square) and that this can outperform classification using filter banks with large support. It is also shown that the performance of filter banks is inferior to that of image patches with equivalent neighborhoods. We develop novel texton-based representations which are suited to modeling this joint neighborhood distribution for Markov random fields. The representations are learned from training images and then used to classify novel images (with unknown viewpoint and lighting) into texture classes. Three such representations are proposed and their performance is assessed and compared to that of filter banks. The power of the method is demonstrated by classifying 2,806 images of all 61 materials present in the Columbia-Utrecht database. The classification performance surpasses that of recent state-of-the-art filter bank-based classifiers such as Leung and Malik (IJCV 01), Cula and Dana (IJCV 04), and Varma and Zisserman (IJCV 05). We also benchmark performance by classifying all of the textures present in the UIUC, Microsoft Textile, and San Francisco outdoor data sets. We conclude with discussions on why features based on compact neighborhoods can correctly discriminate between textures with large global structure and why the performance of filter banks is not superior to that of the source image patches from which they were derived.
INDEX TERMS
Material classification, 3D textures, textons, image patches, filter banks.
CITATION
Manik Varma, Andrew Zisserman, "A Statistical Approach to Material Classification Using Image Patch Exemplars", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 11, pp. 2032-2047, November 2009, doi:10.1109/TPAMI.2008.182
REFERENCES
[1] http://www.robots.ox.ac.uk/~vgg/research/ texclass/datacuret col.zip, 2008.
[2] http://www.robots.ox.ac.uk/~vgg/research/ texclassfilters. html, 2008.
[3] J.R. Bergen and E.H. Adelson, “Early Vision and Texture Perception,” Nature, vol. 333, pp. 363-364, May 1988.
[4] J. Bi, K. Bennett, M. Embrechts, M. Breneman, and M. Song, “Dimensionality Reduction via Sparse Support Vector Machines,” J. Machine Learning Research, vol. 3, pp. 1229-1243, 2003.
[5] C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
[6] R.E. Broadhurst, “Statistical Estimation of Histogram Variation for Texture Classification,” Proc. Fourth Int'l Workshop Texture Analysis and Synthesis, pp. 25-30, Oct. 2005.
[7] A. Buades, B. Coll, and J.-M. Morel, “A Non-Local Algorithm for Image Denoising,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 60-65, June 2005.
[8] C.J.C. Burges, “Geometric Methods for Feature Extraction and Dimensionality Reduction,” Data Mining and Knowledge Discovery Handbook, O. Maimon and L. Rokach, eds., pp. 59-92, Springer, 2005.
[9] B. Caputo, E. Hayman, and P. Mallikarjuna, “Class-Specific Material Categorisation,” Proc. 10th Int'l Conf. Computer Vision, vol. 2, pp. 1597-1604, Oct. 2005.
[10] G.K. Cross and A.K. Jain, “Markov Random Field Texture Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 5, no. 1, pp. 25-39, Nov. 1983.
[11] O.G. Cula and K.J. Dana, “3D Texture Recognition Using Bidirectional Feature Histograms,” Int'l J. Computer Vision, vol. 59, no. 1, pp. 33-60, Aug. 2004.
[12] K.J. Dana, B. van Ginneken, S.K. Nayar, and J.J. Koenderink, “Reflectance and Texture of Real World Surfaces,” ACM Trans. Graphics, vol. 18, no. 1, pp. 1-34, Jan. 1999.
[13] P. Domingos and M.J. Pazzani, “On the Optimality of the Simple Bayesian Classifier under Zero-One Loss,” Machine Learning, vol. 29, nos. 2/3, pp. 103-130, Nov. 1997.
[14] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. John Wiley & Sons, 2001.
[15] A. Efros and T. Leung, “Texture Synthesis by Non-Parametric Sampling,” Proc. Seventh Int'l Conf. Computer Vision, vol. 2, pp.1039-1046, Sept. 1999.
[16] C. Fowlkes, D. Martin, and J. Malik, “Learning Affinity Functions for Image Segmentation: Combining Patch-Based and Gradient-Based Approaches,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 54-61, June 2003.
[17] D.D. Garber, “Computational Models for Texture Analysis and Texture Synthesis,” PhD thesis, Univ. of Southern California, 1981.
[18] S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 6, pp. 721-741, Nov. 1984.
[19] B. Georgescu, I. Shimshoni, and P. Meer, “Mean Shift Based Clustering in High Dimensions: A Texture Classification Example,” Proc. Ninth Int'l Conf. Computer Vision, vol. 1, pp. 456-463, Oct. 2003.
[20] H. Greenspan, S. Belongie, P. Perona, and R. Goodman, “Rotation Invariant Texture Recognition Using a Steerable Pyramid,” Proc. 12th Int'l Conf. Pattern Recognition, vol. 2, pp. 162-167, Oct. 1994.
[21] I. Guyon and A. Elisseeff, “An Introduction to Variable and Feature Selection,” J. Machine Learning Research, vol. 3, pp. 1157-1182, 2003.
[22] G.M. Haley and B.S. Manjunath, “Rotation-Invariant Texture Classification Using Modified Gabor Filters,” Proc. Second IEEE Int'l Conf. Image Processing, vol. 1, pp. 262-265, Oct. 1995.
[23] E. Hayman, B. Caputo, M. Fritz, and J.-O. Eklundh, “On the Significance of Real-World Conditions for Material Classification,” Proc. Eighth European Conf. Computer Vision, vol. 4, pp. 253-266, May 2004.
[24] J. Hays, M. Leordeanu, A. Efros, and Y. Liu, “Discovering Texture Regularity as a Higher-Order Correspondence Problem,” Proc. Ninth European Conf. Computer Vision, vol. 2, pp. 522-535, 2006.
[25] A.K. Jain and K. Karu, “Learning Texture Discrimination Masks,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 2, pp. 195-205, Feb. 1996.
[26] B. Julesz, “Textons, the Elements of Texture Perception, and Their Interactions,” Nature, vol. 290, pp. 91-97, 1981.
[27] B. Julesz, E.N. Gilbert, L.A. Shepp, and H.L. Frisch, “Inability of Humans to Discriminate between Visual Textures that Agree in Second-Order Statistics—Revisited,” Perception, vol. 2, no. 4, pp.391-405, 1973.
[28] R. Kohavi and G.H. John, “Wrappers for Feature Subset Selection,” Artificial Intelligence, vol. 97, nos. 1/2, pp. 273-324, 1997.
[29] S. Konishi and A.L. Yuille, “Statistical Cues for Domain Specific Image Segmentation with Performance Analysis,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 125-132, June 2000.
[30] S. Lazebnik, C. Schmid, and J. Ponce, “A Sparse Texture Representation Using Local Affine Regions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1265-1278, Aug. 2005.
[31] T. Leung and J. Malik, “Representing and Recognizing the Visual Appearance of Materials Using Three-Dimensional Textons,” Int'l J. Computer Vision, vol. 43, no. 1, pp. 29-44, June 2001.
[32] E. Levina, “Statistical Issues in Texture Analysis,” PhD thesis, Univ. of California at Berkeley, 2002.
[33] S.Z. Li, Markov Random Field Modeling in Image Analysis. Springer-Verlag, 2001.
[34] W. Lin, J. Hays, C. Wu, V. Kwatra, and Y. Liu, “Quantitative Evaluation of Near Regular Texture Synthesis Algorithms,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 427-434, 2006.
[35] T. Lindeberg, “Feature Detection with Automatic Scale Selection,” Int'l J. Computer Vision, vol. 30, no. 2, pp. 77-116, Nov. 1998.
[36] Y. Liu, W. Lin, and J. Hays, “Near Regular Texture Analysis and Manipulation,” ACM Trans. Graphics, vol. 23, no. 3, pp. 368-376, 2004.
[37] J. Malik, S. Belongie, T. Leung, and J. Shi, “Contour and Texture Analysis for Image Segmentation,” Int'l J. Computer Vision, vol. 43, no. 1, pp. 7-27, June 2001.
[38] J. Malik and P. Perona, “Preattentive Texture Discrimination with Early Vision Mechanism,” J. Optical Soc. Am., vol. 7, no. 5, pp. 923-932, May 1990.
[39] K. Popat and R.W. Picard, “Novel Cluster-Based Probability Model for Texture Synthesis, Classification, and Compression,” Proc. SPIE Conf. Visual Comm. and Image Processing, pp. 756-768, Nov. 1993.
[40] T. Randen and J.H. Husoy, “Filtering for Texture Classification: A Comparative Study,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 4, pp. 291-310, Apr. 1999.
[41] M. Riesenhuber and T. Poggio, “Hierarchical Models of Object Recognition in Cortex,” Nature Neuroscience, vol. 2, no. 11, pp.1019-1025, Nov. 1999.
[42] S. Savarese and A. Criminsi, Classification of Folded Textiles, http://research.microsoft.com/vision/cambridge/ recognitionMSRC_MaterialsImageDatabase.zip , Aug. 2004.
[43] C. Schmid, “Constructing Models for Content-Based Image Retrieval,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 39-45, Dec. 2001.
[44] C. Schmid, “Weakly Supervised Learning of Visual Models and Its Application to Content-Based Retrieval,” Int'l J. Computer Vision, vol. 56, no. 1, pp. 7-16, 2004.
[45] B. Scholkopf and A. Smola, Learning with Kernels. MIT Press, 2002.
[46] P. Simard, Y. LeCun, J. Denker, and B. Victorri, “Transformation Invariance in Pattern Recognition—Tangent Distance and Tangent Propagation,” Int'l J. Imaging System and Technology, vol. 11, no. 2, pp. 181-194, 2001.
[47] J.R. Smith and S.F. Chang, “Transform Features for Texture Classification and Discrimination in Large Image Databases,” Proc. First IEEE Int'l Conf. Image Processing, vol. 3, pp. 407-411, Nov. 1994.
[48] K. Tieu and P. Viola, “Boosting Image Retrieval,” Int'l J. Computer Vision, vol. 56, nos. 1/2, pp. 17-36, 2004.
[49] M. Unser, “Local Linear Transforms for Texture Measurements,” Signal Processing, vol. 11, no. 1, pp. 61-79, 1986.
[50] M. Varma, “Statistical Approaches to Texture Classification,” PhD thesis, Univ. of Oxford, Oct. 2004.
[51] M. Varma and R. Garg, “Locally Invariant Fractal Features for Statistical Texture Classification,” Proc. 11th Int'l Conf. Computer Vision, 2007.
[52] M. Varma and A. Zisserman, “Texture Classification: Are Filter Banks Necessary?” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 691-698, June 2003.
[53] M. Varma and A. Zisserman, “Unifying Statistical Texture Classification Frameworks,” Image and Vision Computing, vol. 22, no. 14, pp. 1175-1183, Dec. 2004.
[54] M. Varma and A. Zisserman, “A Statistical Approach to Texture Classification from Single Images,” Int'l J. Computer Vision, 2005.
[55] C. Wallraven, B. Caputo, and A. Graf, “Recognition with Local Features: The Kernel Recipe,” Proc. Ninth Int'l Conf. Computer Vision, vol. 1, pp. 257-264, Oct. 2003.
[56] L. Wolf, X. Huang, I. Martin, and D. Metaxas, “Patch Based Texture Edges and Segmentation,” Proc. Ninth European Conf. Computer Vision, vol. 2, pp. 481-493, May 2006.
[57] Y. Xu, H. Ji, and C. Fermuller, “A Projective Invariant for Textures,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 1932-1939, June 2006.
[58] A. Zalesny and L. Van Gool, “A Compact Model for Viewpoint Dependent Texture Synthesis,” Proc. European Workshop 3D Structure from Multiple Images of Large-Scale Environments, pp.124-143, 2000.
18 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool