We present a unified framework for modeling and solving invariant point pattern matching problems. Invariant features are encoded as potentials in a probabilistic graphical model. By using a specific kind of graph topology, different types of invariant matching models can be implemented via tree-width selection. Models with tree-widths 1, 2, 3 and 4 implement translation, similarity, affine and projective invariant point matching, respectively. The optimal match is then found by exploiting the Markov structure of the graph through the generalized distributive law in a dynamic programming setting. In the absence of noise in the point coordinates, the solutions found are optimal. Our early experiments suggest the approach is robust to outliers and moderate noise.