We propose a new learning algorithm to enhance fault tolerance of multi-layer neural networks (MLNs). This method is based on the idea that strong weights make MLNs sensitive to faults. The purpose of the proposed algorithm is to make weights as small as possible through its training. The evaluation function of the proposed algorithm consists of not only the output error but also the square sum of weights. With the new evaluation, function the learning algorithm minimizes not only output error but also weights. We discussed about the value of parameter to balance effects of these two terms. Next, we apply it to pattern recognition problems. As a result, it is shown that the degradation of recognition ratio is improved.