Neural Networks, IEEE - INNS - ENNS International Joint Conference on (2000)
July 24, 2000 to July 27, 2000
Felix A. Gers , IDSIA
Jürgen Schmidhuber , IDSIA
Long Short-Term Memory (LSTM) can learn algorithms for temporal pattern processing not learnable by alternative recurrent neural networks (RNNs) or other methods such as Hidden Markov Models (HMMs) and symbolic grammar learning (SGL). Here we present tasks involving arithmetic operations on continual input streams that even LSTM cannot solve. However, an LSTM variant based on “forget gates”, a recent extension, has superior arithmetic capabilities and does solve the tasks.
J. Schmidhuber and F. A. Gers, "Neural Processing of Complex Continual Input Streams," Neural Networks, IEEE - INNS - ENNS International Joint Conference on(IJCNN), Como, Italy, 2000, pp. 4557.