Issue No. 02 - February (2009 vol. 31)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2008.77
Nuno Vasconcelos , UCSD, La Jolla
Manuela Vasconcelos , UCSD, La Jolla
Low-complexity feature selection is analyzed in the context of visual recognition. It is hypothesized that high-order dependences of bandpass features contain little information for discrimination of natural images. This hypothesis is characterized formally by the introduction of the concepts of conjunctive interference and decomposability order of a feature set. Necessary and sufficient conditions for the feasibility of low-complexity feature selection are then derived in terms of these concepts. It is shown that the intrinsic complexity of feature selection is determined by the decomposability order of the feature set and not its dimension. Feature selection algorithms are then derived for all levels of complexity and are shown to be approximated by existing information-theoretic methods, which they consistently outperform. The new algorithms are also used to objectively test the hypothesis of low decomposability order through comparison of classification performance. It is shown that, for image classification, the gain of modeling feature dependencies has strongly diminishing returns: best results are obtained under the assumption of decomposability order 1. This suggests a generic law for bandpass features extracted from natural images: that the effect, on the dependence of any two features, of observing any other feature is constant across image classes.
Feature extraction and construction, low complexity, natural image statistics, information theory, feature discrimination versus dependence, image databases, object recognition, texture, perceptual reasoning.
Nuno Vasconcelos, Manuela Vasconcelos, "Natural Image Statistics and Low-Complexity Feature Selection", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 31, no. , pp. 228-244, February 2009, doi:10.1109/TPAMI.2008.77