This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Knowledge Discovery by Inductive Neural Networks
November/December 1999 (vol. 11 no. 6)
pp. 992-998

Abstract—A new neural network model for inducing symbolic knowledge from empirical data is presented. This model capitalizes on the fact that the certainty-factor-based activation function can improve the network generalization performance from a limited amount of training data. The formal properties of the procedure for extracting symbolic knowledge from such a trained neural network are investigated. In the domain of molecular genetics, a case study demonstrated that the described learning system effectively discovered the prior domain knowledge with some degree of refinement. Also, in cross-validation experiments, the system outperformed C4.5, a commonly used rule learning system.

[1] R. Andrews, J. Diederich, and A.B. Tickle, “Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks,” Knowledge-Based Systems, vol. 8, no. 6, pp. 373-389, 1995.
[2] E.B. Baum and D. Haussler, “What Size Net Gives Valid Generalization?” Neural Computation, vol. 1, pp. 151-160, 1989.
[3] A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, "Learnability and the Vapnik-Chervonenkis Dimension," J. ACM, vol. 36, pp. 929-965, 1989.
[4] Rule-Based Expert Systems, B.G. Buchanan and E.H. Shortliffe, eds. Reading, Mass.: Addison-Wesley, 1984.
[5] L.M. Fu, "Knowledge-Based Connectionism for Revising Domain Theories," IEEE Trans. Systems, Man, and Cybernetics, vol. 23, no. 1, pp. 173-182, 1993.
[6] L.M. Fu, “Rule Generation from Neural Networks,” IEEE Trans. Systems, Man, and Cybernetics, vol. 24, no. 8, pp. 1,114-1,124, 1994.
[7] L.M. Fu, Neural Networks in Computer Intelligence.McGraw-Hill, 1994.
[8] L.M. Fu, “Learning in Certainty Factor Based Multilayer Neural Networks for Classification,” IEEE Trans. Neural Networks, vol. 9, no. 1, pp. 151-158, 1998.
[9] D.R. Hush and B.G. Horne, "Progress in Supervised Neural Networks," IEEE Signal Processing Magazine, Vol. 10, No. 1, Jan. 1993, pp. 8-38.
[10] J.R. Quinlan, C4.5: Programs for Machine Learning,San Mateo, Calif.: Morgan Kaufman, 1992.
[11] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning Internal Representations by Error Propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1: Foundations, D.E. Rumelhart and J.L. McClelland et al., eds., chapter 8, pp. 318-362.Cambridge, Mass.: MIT Press, 1986.
[12] K. Saito and R. Nakano, "Medical Diagnostic Expert System Based on PDP Model," Proc. IEEE Int'l Conf. Neural Networks, vol. 1, pp. 255-262, 1988.
[13] G.G. Towell, J.W. Shavlik, and M.O. Noordewier, “Refinement of Approximate Domain Theories by Knowledge-Based Neural Networks,” Proc. AAAI '90, pp. 861-866, Boston, 1990.
[14] G.G. Towell and J.W. Shavlik, "Knowledge-Based Artificial Neural Networks," Artificial Intelligence, vol. 70, nos. 1-2, pp. 119-165, 1994.
[15] L.G. Valiant, “A Theory of the Learnable,” Comm. ACM, vol. 27, no. 11, pp. 1134-1142, Nov. 1984.
[16] V.N. Vapnik and A.Y. Chervonenkis, “On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities,” Theory Probability Application, vol. 16, no. 2, pp. 264–280, 1971.

Index Terms:
Knowledge discovery, neural network, rule extraction, machine learning, certainty factor.
Citation:
LiMin Fu, "Knowledge Discovery by Inductive Neural Networks," IEEE Transactions on Knowledge and Data Engineering, vol. 11, no. 6, pp. 992-998, Nov.-Dec. 1999, doi:10.1109/69.824623
Usage of this product signifies your acceptance of the Terms of Use.