In this paper, we present a new scheme for face recognition. The main idea is to represent the images with the similarity features against the reference set and to provide the relative match for two images. For any image, we first compute the similarities between it and all the reference images, and then we take these similarities as its feature. Based on the similarity features, a linear discriminating classifier is constructed to recognize the querying image. Inspired by research in cognitive psychology, the perceptual distance based dynamic similarity function is proposed to compute the similarity features. The proposed method can be regarded as a generalization of kernel discriminant analysis, and it can well deal with the nonlinear variations, especially occlusion. Extensive experiments are conducted to show its performance and robustness to occlusion.