One of the most important tasks in medical image analysis is to detect the absence or presence of disease in an image, without having precise delineations of pathology available for training. A novel method is proposed to solve such a classification task, based on a generalized representation of an image derived from local per-pixel features. From this representation, differences between images can be computed, and these can be used to classify the image requiring knowledge of only global image labels for training. It is shown how to construct multiple representations of one image to get multiple classification opinions and combine them to smooth over errors of individual classifiers. The performance of the method is evaluated on the detection of interstitial lung disease on standard chest radiographs. The best result is obtained for the combining classification scheme yielding an area under the ROC curve of 0.955.