We present a new approach to appearance-based object recognition, which captures the relationships between multiple model views and exploits them to improve recognition performance.

The basic building block are local, viewpoint invariant regions. We propose an efficient algorithm for partitioning a set of region matches into groups lying on smooth surfaces (GAMs). During modeling, the model views are connected by a large number of region-tracks, each aggregating image regions of a single physical region across the views. At recognition time, GAMs are constructed matching a test image to each model view. The consistency of configurations of GAMs is measured by exploiting the model connections. The most consistent configuration, covering the object as completely as possible is found by a genetic algorithm. Introducing GAMs as an intermediate grouping level facilitates decision-making and improves discriminative power.

As a complementary application, we introduce a novel GAM-based two-view filter and demonstrate its effectiveness in recovering correct matches in the presence of up to 96% mismatches.