Issue No. 05 - May (2012 vol. 34)
Nikhil Rasiwasia , University of California at San Diego, La Jolla
Nuno Vasconcelos , University of California at San Diego, La Jolla
A novel framework to context modeling based on the probability of co-occurrence of objects and scenes is proposed. The modeling is quite simple, and builds upon the availability of robust appearance classifiers. Images are represented by their posterior probabilities with respect to a set of contextual models, built upon the bag-of-features image representation, through two layers of probabilistic modeling. The first layer represents the image in a semantic space, where each dimension encodes an appearance-based posterior probability with respect to a concept. Due to the inherent ambiguity of classifying image patches, this representation suffers from a certain amount of contextual noise. The second layer enables robust inference in the presence of this noise by modeling the distribution of each concept in the semantic space. A thorough and systematic experimental evaluation of the proposed context modeling is presented. It is shown that it captures the contextual “gist” of natural images. Scene classification experiments show that contextual classifiers outperform their appearance-based counterparts, irrespective of the precise choice and accuracy of the latter. The effectiveness of the proposed approach to context modeling is further demonstrated through a comparison to existing approaches on scene classification and image retrieval, on benchmark data sets. In all cases, the proposed approach achieves superior results.
Computer vision, scene classification, context, image retrieval, topic models.
Nikhil Rasiwasia, Nuno Vasconcelos, "Holistic Context Models for Visual Recognition", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 34, no. , pp. 902-917, May 2012, doi:10.1109/TPAMI.2011.175