Given a set of patterns and a similarity measure between them, we will present an optimization framework to approximate a small subset, known as a canonical set, whose members closely resemble the members of the original set. We will present a combinatorial formulation of the canonical set problem in terms of quadratic optimization integer programming, present a relaxation through semidefinite programming, and propose a bounded performance rounding procedure for its approximation solution in polynomial time. Through a set of experiments we will investigate the application of canonical sets for computing a summary of views from a dense set of 2D views computed for a 3D object.