This paper presents a model-based object recognition approach that uses a hierarchical Gabor wavelet representation. The key idea is to use magnitude, phase and frequency measures of Gabor wavelet representation in an innovative flexible matching approach that can provide robust recognition. A Gabor grid a topology-preserving map, efficiently encodes both signal energy and structural information of an object in a sparse multi-resolution representation. The Gabor grid subsamples the Gabor wavelet decomposition of an object model and is deformed to allow the indexed object model match with the image data. Flexible matching between the model and the image minimizes a cost function based on local similarity and geometric distortion of the Gabor grid. Grid erosion and repairing is performed whenever a collapsed grid, due to object occlusion, is detected. The results on infrared imagery are presented. Where objects undergo rotation, translation, scale, occlusion and aspect variations under changing environmental conditions.