In this paper we propose an alternative interpretation of Bayesian learning based on maximal evidence principle. We establish a notion of local evidence which can be viewed as a compromise between accuracy of obtained solution with respect to the training sample and its stability with respect to weight changes. The modification of traditional Bayesian approach allows selecting best solution among different models. This methodology was used successfully for choosing best kernel function in relevance vector machines algorithm. Both classification and regression cases are considered.