This Article 
 Bibliographic References 
 Add to: 
Multiresolution Histograms and Their Use for Recognition
July 2004 (vol. 26 no. 7)
pp. 831-847

Abstract—The histogram of image intensities is used extensively for recognition and for retrieval of images and video from visual databases. A single image histogram, however, suffers from the inability to encode spatial image variation. An obvious way to extend this feature is to compute the histograms of multiple resolutions of an image to form a multiresolution histogram. The multiresolution histogram shares many desirable properties with the plain histogram including that they are both fast to compute, space efficient, invariant to rigid motions, and robust to noise. In addition, the multiresolution histogram directly encodes spatial information. We describe a simple yet novel matching algorithm based on the multiresolution histogram that uses the differences between histograms of consecutive image resolutions. We evaluate it against five widely used image features. We show that with our simple feature we achieve or exceed the performance obtained with more complicated features. Further, we show our algorithm to be the most efficient and robust.

[1] J. Bach, C. Fuler, A. Gupta, A. Hampapur, B. Horowitz, R. Humphrey, R. Jain, and C. Shu, The Virage Image Search Engine: An Open Framework for Image Management Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases IV, vol. 2670, pp. 76-87, 1996.
[2] D. Ballard and L. Wixson, Object Recognition Using Steerable Filters at Multiple Scales Proc. IEEE Workshop Qualitative Vision, pp. 2-10, 1993.
[3] A. Barron, Entropy and the Central Limit Theorem The Annals of Probability, vol. 14, no. 1, pp. 336-342, 1986.
[4] N. Blackman, The Convolution Inequality for Entropy Powers IEEE Trans. Information Theory, vol. 11, pp. 267-271, 1965.
[5] J. Bonet, P. Viola, and J. Fisher, Flexible Histograms: A Multiresolution Target Discrimination Model Proc. of SPIE, vol. 3370, 1998.
[6] J.D. Bonet, Multiresolution Sampling Procedure for Analysis and Synthesis of Texture Images Proc. Ann. Conf. Computer Graphics (SIGGRAPH), pp. 361-368, 1997.
[7] A.C. Bovik,M. Clark,, and W.S. Geisler,“Multichannel texture analysis using localized spatial filters,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, pp. 55-73, 1990.
[8] P. Brodatz, Textures: A Photographic Album for Artists and Designers. New York: Dover Publications, 1966.
[9] P.J. Burt and E.H. Adelson, “The Laplacian Pyramid as a Compact Image Code,” IEEE Trans. Comm., vol. 31, no. 4, pp. 532-540, 1983.
[10] F. Campbell and J. Robson, Application of Fourier Analysis to the Visibility of Gratings J. Physiology, vol. 197, pp. 551-566, 1968.
[11] E. Carlen and A. Soffet, Entropy Production by Block Variable Summation and Central Limit Theorems Comm. Math. Physics, vol. 140, pp. 339-371, 1991.
[12] M. Carlotto, Texture Classification Based on Hypothesis Testing Approach Proc. Int'l Japanese Conf. Pattern Recognition, pp. 93-96, 1984.
[13] M. Carlotto, Histogram Analysis Using a Scale-Space Approach IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 1, pp. 121-129, 1987.
[14] T. Chang and C.-C.J. Kuo, "Texture Analysis and Classification With Tree-Structured Wavelet Transform," IEEE Trans. Image Processing, vol. 2, no. 4, pp. 429-441, Oct. 1993.
[15] J. Coggins and A. Jain, A Spatial Filtering Approach to Texture Analysis Pattern Recognition Letters, vol. 3, pp. 195-203, 1985.
[16] K. Dana, B. Ginneken, S. Nayar, and J. Koenderink, Reflectance and Texture of Real-World Surfaces ACM Trans. Graphics, vol. 18, no. 1, pp. 1-34, 1999.
[17] L. Devroye and L. Gyorfi, Nonparametric Density Estimation: The$L_1$View. John Wiley and Sons, 1985.
[18] J. Engel, The Multiresolution Histogram Metrika, vol. 46, pp. 41-57, 1997.
[19] A. Evans, N. Thacker, and J. Mayhew, The Use of Geometric Histograms for Model-Based Object Recognition Proc. British Machine Vision Conf., pp. 429-438, 1993.
[20] O. Faugeras, Texture Analysis and Classification Using a Human Visual Model Proc. Fourth Int'l Conf. Pattern Recognition, pp. 549-552, 1978.
[21] I. Fogel and D. Sagi, Gabor Filters as Texture Discriminator Biological Cybernetics, vol. 61, pp. 103-113, 1989.
[22] B. Funt and G. Finlayson, "Color Constant Color Indexing," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 5, pp. 522-529, May 1995.
[23] H. Greenspan, S. Belongie, R. Goodman, and P. Perona, Rotation Invariant Texture Recognition Using a Steerable Pyramid Proc. 12th IAPR Int'l Conf. Pattern Recognition, vol. 2, pp. 162-167, 1994.
[24] L. Griffin, Scale Imprecision Space Image and Vision Computing, vol. 15, pp. 369-398, 1997.
[25] E. Hadjidemetriou, Use of Histograms for Recognition. Columbia Univ., thesisHadjidemetriou.pdf, 2002.
[26] E. Hadjidemetriou, M. Grossberg, and S. Nayar, Histogram Preserving Image Transformations Int'l J. Computer Vision, vol. 45, no. 1, pp. 5-23, 2001.
[27] E. Hadjidemetriou, M. Grossberg, and S. Nayar, Spatial Information in Multiresolution Histograms Proc. Computer Vision and Pattern Recognition Conf., vol. 1, pp. 702-709, 2001.
[28] J. Hafner, H.S. Sawhney, W. Equitz, M. Flickner, and W. Niblack, “Efficient Color Histogram Indexing for Quadratic Form Distance Functions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 7, pp. 729-736, July 1995.
[29] G.M. Haley and B.S. Manjunath, Rotation-Invariant Texture Classification Using a Complete Space-Frequency Model IEEE Trans. Image Processing, vol. 8, no. 2, pp. 255-269, 1999.
[30] R. Haralick, Statistical and Structural Approaches to Texture Proc. IEEE, vol. 67, pp. 786-804, 1979.
[31] D. Heeger and J. Bergen, Pyramid-Based Texture Analysis/Synthesis Proc. Ann. Conf. Computer Graphics (SIGGRAPH), pp. 229-234, 1995.
[32] C. Helstrom, Probability and Stochastic Processes for Engineers. Toronto: Macmillan, 1991.
[33] W. Hsu, T. Chua, and H. Pung, An Integrated Color-Spatial Approach to Content-Based Image Retrieval Proc. ACM Multimedia, pp. 305-313, 1995.
[34] J. Huang, S. Kumar, M. Mitra, W. Zhu, and R. Zabih, Spatial Color Indexing and Applications Int'l J. Computer Vision, vol. 35, no. 3, pp. 245-268, 1999.
[35] B. Huet and E.R. Hancock, “Line Pattern Retrieval Using Relational Histograms,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 12, pp. 1,363-1,371, Dec. 1999.
[36] M. Jägersand, “Saliency Maps and Attention Selection in Scale and Spatial Coordinates: An Information Thoretic Approach,” Proc. Fifth Int'l Conf. Computer Vision, pp. 195-202, 1995.
[37] A. Jain and F. Farrokhnia, Unsupervised Texture Segmentation Using Gabor Filters Pattern Recognition, vol. 24, no. 12, pp. 1167-1186, 1991.
[38] A. Jain, N. Ratha, and S. Lakshmanan, Object Detection Using Gabor Filters Pattern Recognition, vol. 30, no. 2, pp. 295-309, 1997.
[39] O. Johnson, Entropy Inequalities and the Central Limit Theorem Stochastic Processes and Their Applications, vol. 88, pp. 291-304, 2000.
[40] T. Kadir and M. Brady, Saliency, Scale and Image Description Int'l J. Computer Vision, vol. 45, no. 2, pp. 83-105, 2001.
[41] J. Koenderink, The Structure of Images Biological Cybernetics, vol. 50, pp. 363-370, 1984.
[42] J. Koenderink and A.J. van Doorn, The Structure of Locally Orderless Images Int'l J. Computer Vision, vol. 31, nos. 2-3, pp. 159-168, 1999.
[43] A. Laine and J. Fan, “Texture Classification by Wavelet Packet Signature,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 11, pp. 1,186-1,191, Nov. 1993.
[44] S. Lakshmanan and H. Derin, Gaussian Markov Random Fields at Multiple Resolutions Markov Random Fields: Theory and Application, R. Chellapa and A. Jain, eds., pp. 131-157, 1993.
[45] J. Lee and B. Dickinson, Multiresolution Video Indexing for Subband Coded Video Databases Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases II, vol. 2185, pp. 162-173, 1994.
[46] W. Leow and R. Li, Adaptive Binning and Dissimilarity Measure for Image Retrieval and Classification Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 234-239, 2001.
[47] G. Lowitz, Can a Local Histogram Really Map Texture Information? Pattern Recognition, vol. 16, pp. 141-147, 1983.
[48] J. Malik and P. Perona, Preattentive Texture Discrimination with Early Vision Mechanisms J. Optical Soc. Am. A, vol. 7, no. 5, pp. 923-932, 1990.
[49] R. Manthalkar, P. Biswas, and B. Chatterji, Rotation and Scale Invariant Texture Classification Using Gabor Wavelets Proc. Second Int'l Workshop Texture Analysis and Synthesis (in conjuction with ECCV), pp. 87-90, 2002.
[50] J. Mao and A. Jain, Texture Classification and Segmentation Using Multiresolution Simultaneous Autoregressive Models Pattern Recognition, vol. 25, no. 2, pp. 173-188, 1992.
[51] B. Mel, Combining Color Shape and Texture Histogramming in a Neurally Inspired Approach to Visual Object Recognition Neural Computation, vol. 9, pp. 777-804, 1997.
[52] W. Niblack, The QBIC Project: Querying Images by Content Using Color, Texture, and Shape Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases, vol. 1908, pp. 173-187, 1993.
[53] B. Noble and J. Daniel, Applied Linear Algebra. Prentice-Hall, 1988.
[54] A. Oomes and P. Snoeren, Structural Information in Scale-Space Proc. Copenhagen Workshop Gaussian Scale-Space Theory, vol. 96, no. 19, pp. 48-57, May 1996.
[55] G. Pass and R. Zabih, Histogram Refinement for Content Based Image Retrieval Proc. IEEE Workshop Applications of Computer Vision, pp. 96-102, 1996.
[56] G. Pass and R. Zabih, Comparing Images Using Joint Histograms Multimedia Systems, vol. 7, no. 3, pp. 234-240, 1999.
[57] F. Pennini and A. Plastino, Fisher's Information Measure in a Tsallis' Nonextensive Setting and Its Application to Diffusive Processes Physica A, vol. 247, pp. 559-569, 1997.
[58] A. Plastino, A. Plastino, and H. Miller, Tsallis Nonextensive Thermostatistics and Fisher's Information Measure Physica A, vol. 235, pp. 577-588, 1997.
[59] M. Popovic, Texture Analysis Using 2D Wavelet Transform: Theory and Applications Proc. IEEE TELSIKS, pp. 149-158, 1999.
[60] M. Porat and Y. Zeevi, "The Generalized Gabor Scheme of Image Representation in Biological and Machine Vision," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 4, pp. 452-468, 1988.
[61] J. Puzicha, T. Hofmann, and J. Buchmann, Histogram Clustering for Unsupervised Segmentation and Image Retrieval Pattern Recognition Letters, vol. 20, no. 9, pp. 899-909, 1999.
[62] S. Ravela and R. Manmatha, Retrieving Images by Appearance Proc. Int'l Conf. Computer Vision, pp. 608-613, 1998.
[63] Y. Rubner, L. Guibas, and C. Tomasi, The Earth Mover's Distance, Multi-Dimensional Scaling, and Color-Based Image Retrieval Proc. ARPA Image Understanding Workshop, 1997.
[64] S. Sablak and T. Boult, Multilevel Color Histogram Representation of Color Images by Peaks for Omni-Camera Proc. IASTED Int'l Conf. Signal and Image Processing, 1999.
[65] B. Schiele and J. Crowley, Recognition without Correspondence Using Multidimensional Receptive Field Histograms Int'l J. Computer Vision, vol. 36, no. 1, pp. 31-50, 2000.
[66] D. Scott, On Optimal and Data-Based Histograms Biometrica, vol. 66, no. 3, pp. 605-610, 1979.
[67] J. Smith and S. Chang, Tools and Techniques for Color Image Retrieval Proc. SPIE, vol. 2670, pp. 1630-1639, 1996.
[68] J. Sporring, C. Colios, and P. Trahanias, Generalized Scale-Space Proc. Int'l Conf. Image Processing, vol. 1, pp. 920-923, 2000.
[69] J. Sporring and J. Weickert, “Information Measures in Scale-Spaces,” IEEE Trans. Information Theory, vol. 45, no. 3, pp. 1051-1058, 1999.
[70] A. Stam, Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon Information and Control, vol. 2, pp. 101-112, 1959.
[71] M. Stricker and A. Dimai, Color Indexing with Weak Spatial Constraints Proc. SPIE Storage and Retrieval for Image and Video Databases, pp. 29-39, 1996.
[72] M. Stricker and M. Orengo, Similarity of Color Images Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases III, vol. 2420, pp. 381-392, 1995.
[73] M. Swain and D. Ballard, Color Indexing Int'l J. Computer Vision, vol. 7, no. 1, pp. 11-32, 1991.
[74] M. Tanaka, T. Watanabe, and T. Mishima, Tsallis Entropy in Scale-Spaces Proc. SPIE Conf. Vision Geometry VIII, vol. 3811, pp. 273-281, 1999.
[75] C. Taylor, Akaike's Information Criterion and the Histogram Biometrica, vol. 74, no. 3, pp. 636-639, 1987.
[76] C. Tsallis, Nonextensive Statistics: Theoretical, Experimental and Computational Evidences and Connections Brazilian J. Physics, vol. 29, no. 1, 1999.
[77] J. Weszka, C. Dryer, and A. Rosenfeld, A Comparative Study of Texture Measures for Terrain Classification IEEE Trans. Systems, Man, and Cybernetics, vol. 6, no. 4, pp. 269-285, 1976.
[78] A. Witkin, Scale-Space Filtering Proc. of Int'l Joint Conf. of Artificial Intelligence, pp. 1019-1022, 1983.
[79] R. Young, The Gaussian Derivative Theory of Spatial Vision: Analysis of Cortical Cell Receptive Field Line Weighting Profiles Technical Report GMR 4920, 1985.
[80] H. Zhang, A. Kankanhali, and S. Smoliar, Automatic Partitioning of Full-Motion Video Multimedia Systems, vol. 1, pp. 10-28, 1993.
[81] H. Zhang, C. Low, W. Smoliar, and J. Wu, Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution ACM Multimedia, pp. 15-24, 1995.

Index Terms:
Multiresolution histogram, scale-space, image sharpness, Fisher information, shape feature, texture feature, histogram matching, histogram bin width, feature parameter sensitivity, feature comparison.
Efstathios Hadjidemetriou, Michael D. Grossberg, Shree K. Nayar, "Multiresolution Histograms and Their Use for Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 7, pp. 831-847, July 2004, doi:10.1109/TPAMI.2004.32
Usage of this product signifies your acceptance of the Terms of Use.