We propose a learning algorithm to enhance the fault tolerance of feedforward neural networks (NNs for short) by manipulating the gradient of sigmoid activation function of the neuron. For the output layer, we employ the function with the relatively gentle gradient. For the hidden layer, we steepen the gradient of function after convergence. The experimental results show that our NNs are superior to NNs trained with other algorithms employing fault injection and the calculation of relevance of each weight to the output error in fault tolerance, learning cycles and time. Besides our gradient manipulation never spoils the generalization ability.