This paper addresses fault tolerant architecture of multi-layer neural networks with a genetic algorithm scheme. For large scale neural networks implemented in single chip or silicon wafer, it is necessary to develop self-recovery mechanisms that can automatically recover faults without a host computer. In this paper, we propose a fault tolerant multi-layer neural networks employing both hardware redundancy and weight retraining in order to realize self-recovering neural networks. The main advantages of our architecture are low hardware cost for adding redundant neurons and fast training by a genetic algorithm implemented in hardware. A prototype system is implemented on a field programmable gate array to show the possibility of self-recovering neural networks.