Diagnosis algorithms for integrated circuits (ICs) are typically developed and evaluated using a limited number of logic-level models of defect behaviors. However, it is well-known that real IC defects exhibit behavior well outside these models. Consequently, the utility of IC diagnosis methodologies may be uncertain. In this paper, a simulation-based benchmarking strategy is developed that uses circuit-level models to describe the complex nature of real defects. Specifically, we have proposed a simple yet powerful strategy using a small circuit and a set of bounded deformations (i.e., defects) for measuring the effectiveness of diagnosis techniques. Evaluation of several simple and commercial diagnosis algorithms indicates that this form of diagnosis benchmarking is viable.