We propose a framework incorporating aspects of image classification to aid the matching of a reference image to target images. The framework involves an image representation based on a set of feature vectors, and a parametric distance measure on any two such vector sets. The distance measure may be optimized to provide maximum discrimination between the matching target images and the background images, when compared to the reference image. Preliminary results indicate that the new distance measure performs substantially better than the traditional SSD and the Bhattacharyya histogram measures in classification and tracking tasks.