This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Context-Based Object-Class Recognition and Retrieval by Generalized Correlograms
October 2007 (vol. 29 no. 10)
pp. 1818-1833
We present a novel approach for retrieval of object categories based on a novel type of image representation: the Generalized Correlogram (GC). In our image representation, the object is described as a constellation of GCs where each one encodes information about some local part and the spatial relations from this part to others (i.e., the part's context). We show how such a representation can be used with fast procedures that learn the object category with weak supervision and efficiently match the model of the object against large collections of images. In the learning stage, we show that by integrating our representation with Boosting the system is able to obtain a compact model that is represented by very few features, where each feature conveys key properties about the object's parts and their spatial arrangement. In the matching step, we propose direct procedures that exploit our representation for efficiently considering spatial coherence between the matching of local parts. Combined with an appropriate data organization such as Inverted Files, we show that thousands of images can be evaluated efficiently. The framework has been applied to different standard databases and we show that our results are favorably compared against state-of-the-art methods in both computational cost and accuracy

[1] A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-Based Image Retrieval at the End of the Early Years,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, Dec. 2000.
[2] P. Enser, “Query Analysis in a Visual Information Retrieval Context,” J. Documentation and Text Management, vol. 1, pp. 25-52, 1993.
[3] M. Markkula and E. Sormunen, “End-User Searching Challenges Indexing Practices in the Digital Newspaper Photo Archive,” Information Retrieval, vol. 1, pp. 259-285, 2000.
[4] R. Fergus, P. Perona, and A. Zisserman, “Object Class Recognition by Unsupervised Scale-Invariant Learning,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2003.
[5] L. Fei-Fei, R. Fergus, and P. Perona, “Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition Workshop Generative-Model Based Vision, 2004.
[6] A. Torralba, K.P. Murphy, and W.T. Freeman, “Sharing Features: Efficient Boosting Procedures for Multiclass Object Detection,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 762-769, 2004.
[7] R. Marée, P. Geurts, and J. Piater, “Random Subwindows for Robust Image Classification,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 34-40, 2005.
[8] Y. Li, L.G. Shapiro, and J.A. Bilmes, “A Generative/Discriminative Learning Algorithm for Image Classification,” Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1605-1612, 2005.
[9] A. Pentland, R.W. Picard, and S. Sclaroff, “Photobook—Content-Based Manipulation of Image Databases,” Int'l J. Computer Vision, vol. 18, no. 3, pp. 233-254, 1996.
[10] P. Hong and T.S. Huang, “Spatial Pattern Discovery by Learning a Probabilistic Parametric Model from Multiple Attributed Relational Graphs,” J. Discrete Applied Math., vol. 139, nos. 1-3, pp. 113-135, Apr. 2003.
[11] M. Weber, M. Welling, and P. Perona, “Towards Automatic Discovery of Object Categories,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 101-108, 2000.
[12] D. Crandall, P. Felzenszwalb, and D. Huttenlocher, “Spatial Priors for Part-Based Recognition Using Statistical Models,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[13] P. Viola and M.J. Jones, “Robust Real-Time Face Detection,” Int'l J. Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
[14] J. Amores, N. Sebe, and P. Radeva, “Fast Spatial Pattern Discovery Integrating Boosting with Constellations of Contextual Descriptors,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 769-774, 2005.
[15] J.Z. Wang, J. Li, and G. Wiederhold, “SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 9, pp. 947-963, Sept. 2001.
[16] C. Carson, S. Belongie, H. Greenspan, and J. Malik, “Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 8, pp. 1026-1038, Aug. 2002.
[17] D. Forsyth, J. Malik, and R. Wilensky, “Searching for Digital Pictures,” Scientific Am., vol. 276, no. 6, pp. 72-77, 1997.
[18] Y. Chen and J. Wang, “A Region-Based Fuzzy Feature Matching Approach to Content-Based Image Retrieval,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1252-1267, Sept. 2002.
[19] A. Bar-Hillel, T. Hertz, and D. Weinshall, “Efficient Learning of Relational Object Class Models,” Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1762-1769, 2005.
[20] S. Agarwal, A. Awan, and D. Roth, “Learning to Detect Objects in Images via a Sparse, Part-Based Representation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 11, pp. 1475-1490, Nov. 2004.
[21] R.C. Nelson and A. Selinger, “Large-Scale Tests of a Keyed, Appearance-Based 3D Object Recognition System,” Vision Research, vol. 38, no. 15, pp. 2469-2488, 1998.
[22] S.L.C. Schmid and J. Ponce, “Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2169-2178, 2006.
[23] T. Serre, L. Wolf, and T. Poggio, “Object Recognition with Features Inspired by Visual Cortex,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 994-1000, 2005.
[24] J. Mutch and D. Lowe, “Multiclass Object Recognition with Sparse, Localized Features,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 11-18, 2006.
[25] J. Huang, S.R. Kumar, M. Mitra, W.J. Zhu, and R. Zabih, “Spatial Color Indexing and Applications,” Int'l J. Computer Vision, vol. 35, no. 3, pp. 245-268, 1999.
[26] T. Gevers and A.W.M. Smeulders, “PicToSeek: Combining Color and Shape Invariant Features for Image Retrieval,” IEEE Trans. Image Processing, vol. 9, no. 1, pp. 102-119, Jan. 2000.
[27] Y. Wang, F. Makedon, and A. Chakrabarti, “${\rm R}^{ast}$ -Histograms: Efficient Representation of Spatial Relations between Objects of Arbitrary Topology,” Proc. ACM Multimedia, pp. 356-359, 2004.
[28] S. Belongie, J. Malik, and J. Puzicha, “Shape Matching and Object Recognition Using Shape Contexts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 509-522, Apr. 2002.
[29] H.G. Barrow, J.M. Tenenbaum, R.C. Bolles, and H.C. Wolf, “Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching,” Proc. Int'l Joint Conf. Artificial Intelligence, pp. 659-663, 1977.
[30] Y. Freund and R.E. Schapire, “A Short Introduction to Boosting,” J. Japanese Soc. Artificial Intelligence, 1999.
[31] K. Tieu and P. Viola, “Boosting Image Retrieval,” Int'l J. Computer Vision, vol. 56, nos. 1-2, pp. 17-36, 2004.
[32] J. Friedman, T. Hastie, and R. Tibshirani, “Additive Logistic Regression: A Statistical View of Boosting,” Annals of Statistics, vol. 38, no. 2, pp. 337-374, 2000.
[33] D. Squire, W. Muller, H. Muller, and T. Pun, “Content-Based Query of Image Databases: Inspirations from Text Retrieval,” Pattern Recognition Letters, vol. 21, pp. 1193-1198, 2000.
[34] A. Bar-Hillel and D. Weinshall, “Efficient Learning of Relational Object Class Models,” Int'l J. Computer Vision, pending publication.
[35] S.A. Nene, S.K. Nayar, and H. Murase, “Columbia Object Image Library,” Technical Report CUCS-006-96, Columbia Univ., 1996.
[36] E. Nowak, F. Jurie, and B. Triggs, “Sampling Strategies for Bag-of-Features Image Classification,” Proc. European Conf. Computer Vision, 2006.

Index Terms:
object recognition, retrieval, boosting, spatial pattern, contextual information
Citation:
Jaume Amores, Nicu Sebe, Petia Radeva, "Context-Based Object-Class Recognition and Retrieval by Generalized Correlograms," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1818-1833, Oct. 2007, doi:10.1109/TPAMI.2007.1098
Usage of this product signifies your acceptance of the Terms of Use.