Neural networks have found their place among speech recognition technologies mainly with hybrid models that integrates the time warping ability of Hidden Markov Models (HMM) with the pattern recognition capability of neural networks (NN). Hybrid HMM-NN models have been investigated by several research teams, and constitute now a mature technology highly competitive with HMMs. The authors' contribution to this research field is the introduction of a hybrid HMM-NN model whose original points are a training procedure which employs an integrated gradual movement of bootstrap speech segmentations to shorten training time, a time/feature architecture of the first hidden layer devoted to exploit a better feature selection using some a priori knowledge of speech, the use of a particular kind of acoustical modeling more suitable for a discriminative training, and a method to speed-up the execution of the neural component that makes possible develop real time applications using low cost hardware. Recently, the synergy of several input speech parameterizations has been experimented, leading to a further improvement in the recognition accuracy.