This paper describes a new pulse mode hardware multilayer neural network (MNN) that uses floating-point number system for synapse weights. Combined with pulse mode operation, the floating-point operation is implemented without multiplier. Furthermore, back-propagation algorithm is included in the hardware to provide on-chip learning capability. The proposed MNN is implemented on field programmable gate array (FPGA) and various experiments are conducted to test the performance of the proposed system. The results of the experiments show that the proposed MNN architecture can be used for applications that require high precision in their calculation, and its good on-chip learning capability is demonstrated.