Many medical diagnosis applications are characterized by datasets that contain under-represented classes due to the fact that the disease appears more rarely than the normal case. In such a situation classifiers that generalize over the data such as decision trees and Na?ve Bayesian are not the proper choice as classification methods. Case-based classifiers that can work on the samples seen so far are more appropriate for such a task. We propose to calculate the contingency table and class specific evaluation measures despite the overall accuracy for evaluation purposes of classifiers for these specific data characteristics. We evaluate the different options of our case-based classifier and compare the performance to decision trees and Na?ve Bayesian. Finally, we give an outlook for further work.