This Article 
 Bibliographic References 
 Add to: 
Knowledge Discovery by Inductive Neural Networks
November/December 1999 (vol. 11 no. 6)
pp. 992-998

Abstract—A new neural network model for inducing symbolic knowledge from empirical data is presented. This model capitalizes on the fact that the certainty-factor-based activation function can improve the network generalization performance from a limited amount of training data. The formal properties of the procedure for extracting symbolic knowledge from such a trained neural network are investigated. In the domain of molecular genetics, a case study demonstrated that the described learning system effectively discovered the prior domain knowledge with some degree of refinement. Also, in cross-validation experiments, the system outperformed C4.5, a commonly used rule learning system.

[1] R. Andrews, J. Diederich, and A.B. Tickle, “Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks,” Knowledge-Based Systems, vol. 8, no. 6, pp. 373-389, 1995.
[2] E.B. Baum and D. Haussler, “What Size Net Gives Valid Generalization?” Neural Computation, vol. 1, pp. 151-160, 1989.
[3] A. Blumer, A. Ehrenfeucht, D. Haussler, and M. Warmuth, "Learnability and the Vapnik-Chervonenkis Dimension," J. ACM, vol. 36, pp. 929-965, 1989.
[4] Rule-Based Expert Systems, B.G. Buchanan and E.H. Shortliffe, eds. Reading, Mass.: Addison-Wesley, 1984.
[5] L.M. Fu, "Knowledge-Based Connectionism for Revising Domain Theories," IEEE Trans. Systems, Man, and Cybernetics, vol. 23, no. 1, pp. 173-182, 1993.
[6] L.M. Fu, “Rule Generation from Neural Networks,” IEEE Trans. Systems, Man, and Cybernetics, vol. 24, no. 8, pp. 1,114-1,124, 1994.
[7] L.M. Fu, Neural Networks in Computer Intelligence.McGraw-Hill, 1994.
[8] L.M. Fu, “Learning in Certainty Factor Based Multilayer Neural Networks for Classification,” IEEE Trans. Neural Networks, vol. 9, no. 1, pp. 151-158, 1998.
[9] D.R. Hush and B.G. Horne, "Progress in Supervised Neural Networks," IEEE Signal Processing Magazine, Vol. 10, No. 1, Jan. 1993, pp. 8-38.
[10] J.R. Quinlan, C4.5: Programs for Machine Learning,San Mateo, Calif.: Morgan Kaufman, 1992.
[11] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning Internal Representations by Error Propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1: Foundations, D.E. Rumelhart and J.L. McClelland et al., eds., chapter 8, pp. 318-362.Cambridge, Mass.: MIT Press, 1986.
[12] K. Saito and R. Nakano, "Medical Diagnostic Expert System Based on PDP Model," Proc. IEEE Int'l Conf. Neural Networks, vol. 1, pp. 255-262, 1988.
[13] G.G. Towell, J.W. Shavlik, and M.O. Noordewier, “Refinement of Approximate Domain Theories by Knowledge-Based Neural Networks,” Proc. AAAI '90, pp. 861-866, Boston, 1990.
[14] G.G. Towell and J.W. Shavlik, "Knowledge-Based Artificial Neural Networks," Artificial Intelligence, vol. 70, nos. 1-2, pp. 119-165, 1994.
[15] L.G. Valiant, “A Theory of the Learnable,” Comm. ACM, vol. 27, no. 11, pp. 1134-1142, Nov. 1984.
[16] V.N. Vapnik and A.Y. Chervonenkis, “On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities,” Theory Probability Application, vol. 16, no. 2, pp. 264–280, 1971.

Index Terms:
Knowledge discovery, neural network, rule extraction, machine learning, certainty factor.
LiMin Fu, "Knowledge Discovery by Inductive Neural Networks," IEEE Transactions on Knowledge and Data Engineering, vol. 11, no. 6, pp. 992-998, Nov.-Dec. 1999, doi:10.1109/69.824623
Usage of this product signifies your acceptance of the Terms of Use.