Visual categorization is fundamentally important for autonomous mobile robots to get intelligence such as novel object acquisition and topological place recognition. The main difficulty of visual categorization is how to reduce the large intra-class variations. In this paper, we present a new method made robust to that problem by using intermediate blurring and entropy-guided codebook selection in a bagof- words framework. Intermediate blurring can reduce the high frequency of surface markings and provide dominant shape information. Entropy of a hypothesized codebook can provide the necessary amount of repetition among training exemplars. A generative optimal codebook for each category is learned using the MDL (minimum description length) principle guided by entropy information. Finally, a discriminative codebook is learned using the discriminative method guided by the inter-category entropy of the codebook. We validate the effect of the proposed method using a Caltech-101 DB, which has large intra-class variations.