In this paper, we introduce a Regression Nearest Neighbor framework for general classification tasks. To alleviate potential problems caused by nonlinearity, we propose a kernel regression nearest neighbor (KRNN) algorithmand its convex counterpart (CKRNN) as two specific extensions of nearest neighbor algorithm and present a fast and useful kernel selection method correspondingly. Comprehensive analysis and extensive experiments are used to demonstrate the effectiveness of our methods in real face datasets