In medical diagnosis classification, we often face the unbalanced number of data samples between the classes in which there are not enough samples in rare classes. Conventional competitive learning methods are not suitable in this situation, because they usually tend to be biased to the classes that have the larger number of data samples. In this paper, we proposed a cost-sensitive extension of Regularized Least Square(RLS) algorithm that penalizes errors of different samples with different weights and some rules of thumb to determine those weights. The significantly better classification accuracy of weighted RLS classifiers showed that it is promising substitution of other previous cost-sensitive classification methods for unbalanced data set.