This Article 
 Bibliographic References 
 Add to: 
Unified Integration of Explicit Knowledge and Learning by Example in Recurrent Networks
April 1995 (vol. 7 no. 2)
pp. 340-346

Abstract—We propose a novel unified approach for integrating explicit knowledge and learning by example in recurrent networks. The explicit knowledge is represented by automaton rules, which are directly injected into the connections of a network. This can be accomplished by using a technique based on linear programming, instead of learning from random initial weights. Learning is conceived as a refinement process and is mainly responsible for uncertain information management. We present preliminary results for problems of automatic speech recognition.

[1] E.K. Blum,“Approximation of Boolean functions by sigmoidal networks:Part I: XOR and other two-variable functions,” Neural Computations, vol. 1, no. 4, pp. 532-540, 1989.
[2] H. Bourlard and C. Wellekens,“Speech pattern discrimination and multi-layered perceptrons,” Computer Speech and Language, vol. 3, pp. 1-19, 1989.
[3] A. Cleeremans,D. Servan-Schreiber,, and J.L. McClelland,“Finite state automata and simple recurrent networks,” Neural Computation, vol. 1, no. 3, pp. 372-381, 1989.
[4] P. De Michelis,L. Fissore,P. Laface,G. Micca,, and E. Piccolo,“On the use of neural networks for speaker independent isolated word recognition,” Proc. ICASSP-89,Glasgow, Scotland, May 1989.
[5] J.L. Elman,“Finding structure in time,” CRL Tech. Report 9901. Ctr. for Research in Language,Univ. of California, San Diego, Calif., 1988.
[6] J.L. Elman and D. Zipser,“Learning the hidden structure of the speech,” J. Acoustic Soc. Am., vol. 83, no. 4, pp. 1615-1626, Apr. 1988.
[7] P. Frasconi,M. Gori,M. Maggini,, and G. Soda,“An unified approach for integrating explicit knowledge and learningby example in recurrent networks,” Proc. IEEE-IJCNN91,Seattle, pp. 811-816, 1991.
[8] P. Frasconi,M. Gori,, and G. Soda,“Local feedback multi-layered networks,” Neural Computation, vol. 4, no. 1, pp. 120-130, 1992.
[9] P. Frasconi,M. Gori,, and G. Soda,“Recurrent networks for continuous speech recognition,” Computational Intelligence 90,Milan, Italy: Elsevier, Sept. 1990, pp. 45-56.
[10] M. Gori,Y. Bengio,, and R. De Mori,“BPS: A learning algorithm for capturing the dynamical nature of speech,” Proc. IEEE-IJCNN89,Washington, D.C., 1989.
[11] M. Gori and A. Tesi,“On the problem of local minima in backpropagation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 1, pp. 76-86, 1992.
[12] J.S. Judd,Neural Network Design and the Complexity of Learning,Cambridge, Mass.: MIT Press, 1990.
[13] B.K. Kammerer and W.A. Kupper,“Design of hierarchical perceptron structures and their application to the task of isolated word recognition,” Proc. IEEE-IJCNN89,Washington, D.C., 1989.
[14] W. Hahn,Stability of Motion,New York: Springer Verlag, 1967.
[15] B.A. Pearlmutter,“Learning state space trajectories in recurrent neural networks,” Neural Computation, vol. 1, no. 2, pp. 263-269, 1989.
[16] S.M. Peeling,R.K. Moore,, and M.J. Tomlinson,“The multi-layer perceptron as a tool for speech pattern processing research,” Proc. 10th Autumn Conf. Speech and Hearing, 1986.
[17] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning Internal Representations by Error Propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1: Foundations, D.E. Rumelhart and J.L. McClelland et al., eds., chapter 8, pp. 318-362.Cambridge, Mass.: MIT Press, 1986.
[18] J.E. Hopcroft and J.D. Ullman,Formal Languages and Their Relation to Automata,Reading, Mass.: Addison-Wesley, 1969.
[19] A. Waibel,T. Hanazawa,G. Hinton,K. Shikano,, and K. Lang,“Phoneme recognition using time-delay neural networks,” IEEE Trans. ASSP, vol. 37, no. 3, Mar. 1989.
[20] A. Waibel,“Modularity in neural networks for speech recognition,” Proc. IEEE Conf. Neural Information Processing Systems,Denver, Colo.
[21] R.J. Williams and D. Zipser,“A learning algorithm for continually running fully recurrent networks,” Neural Computation vol. 1, no. 2, pp. 270-280, 1989.
[22] R.J. Williams and J. Peng,“An efficient gradient-based algorithm for on-line training of recurrent networks trajectories,” Neural Computation vol. 2, no. 4, pp. 490-501, 1990.

Index Terms:
Recurrent neural networks, learning automata, automatic speech recognition.
Paolo Frasconi, Marco Gori, Marco Maggini, Giovanni Soda, "Unified Integration of Explicit Knowledge and Learning by Example in Recurrent Networks," IEEE Transactions on Knowledge and Data Engineering, vol. 7, no. 2, pp. 340-346, April 1995, doi:10.1109/69.382304
Usage of this product signifies your acceptance of the Terms of Use.