As object recognition algorithms focus more closely on appearance, the problem of how best to measure similarity between images becomes critical. Within the context of an existing object recognition system, we present an empirical comparison of 27 different error functions on a set of 20 example problems. Each error function measures the similarity between an observed and a predicted image. Our goal is to select an error best suited to guide a heuristic search algorithm through a space of possible scene configurations.