2008 11th IEEE International Conference on Computational Science and Engineering Stabilizing and Improving the Learning Speed of 2-Layered LSTM Network July 16-July 18 ISBN: 978-0-7695-3193-9
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/CSE.2008.32
This paper presents a novel method to initialize the LSTM network weights in order to improve and stabilize the learning speed, based on Nguyen and Widrow’s work for MLP networks. The derived equations for weight initialization are based on the study of the behavior of the memory cells output in the hidden layer. To test and evaluate the proposed method, we use a 2-Layered LSTM network to approximate one and two dimensional real non-linear functions. The obtained results show that our initialization method improves the training process.
Index Terms:
LSTM, neural networks, learning, weight initialization
Citation:
D?bora C. Corr?, Alexandre L. M. Levada, Jos? Hiroki Saito, "Stabilizing and Improving the Learning Speed of 2-Layered LSTM Network," cse, pp.293-300, 2008 11th IEEE International Conference on Computational Science and Engineering, 2008 Usage of this product signifies your acceptance of the Terms of Use. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb