This Article 
 Bibliographic References 
 Add to: 
Generalized Analytic Rule Extraction for Feedforward Neural Networks
November/December 1999 (vol. 11 no. 6)
pp. 985-992

Abstract—This paper suggests the “Input-Network-Training-Output-Extraction-Knowledge” framework to classify existing rule extraction algorithms for feedforward neural networks. Based on the suggested framework, we identify the major practices of existing algorithms as relying on the technique of generate and test, which leads to exponential complexity, relying on specialized network structure and training algorithms, which leads to limited applications and reliance on the interpretation of hidden nodes, which leads to proliferation of classification rules and their incomprehensibility. In order to generalize the applicability of rule extraction, we propose the rule extraction algorithm GeneraLized Analytic Rule Extraction (GLARE), and demonstrate its efficacy by comparing it with neural networks per se and the popular rule extraction program for decision trees, C4.5.

[1] R. Andrews, J. Diederich, and A.B. Tickle, “Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks,” Knowledge-Based Systems, vol. 8, no. 6, pp. 373-389, Dec. 1995.
[2] R. Andrews and S. Geva, “Rule Extraction from a Constrained Error Back Propagation MLP,” Proc. Fifth Australian Conf. Neural Networks, pp. 9-12, 1994.
[3] M.W. Craven and J.W. Shavlik, “Using Sampling and Queries to Extract Rules from Trained Neural Networks,” Proc. 11th Int'l Conf. Machine Learning, pp. 37-45, 1994.
[4] S.E. Decatur, “Application of Neural Networks to Terrain Classification,” Proc. Int'l Joint Conf. Neural Networks, vol. 1, pp. 283-288, 1989.
[5] S. Dutta and S. Shekhar, “Bond-Rating: A Non-Conservative Application of Neural Networks,” Proc. IEEE Int'l Conf. Neural Networks, vol. 2, pp. 443-450, 1988.
[6] L.M. Fu, “Rule Generation from Neural Networks,” IEEE Trans. Systems, Man, and Cybernetics, vol. 24, no. 8, pp. 1,114-1,124, 1994.
[7] S. Gallant,“Connectionist expert systems,” Communication of ACM, v.31(2), pp. 152-169, 1988.
[8] H. Lu, R. Setiono, and H. Liu, Effective Data Mining Using Neural Networks IEEE Trans. Knowledge and Data Eng., vol. 8, no. 6, Dec. 1996.
[9] C. McMillan, M.C. Mozer, and P. Smolensky, “The Connectionist Scientist Game: Rule Extraction and Refinement in a Neural Network,” Proc. 13th Ann. Conf. Cognitive Science Soc., pp. 424-430, 1991.
[10] P.M. Murphy and D.W. Aha, UCI Repository of Machine Learning Databases, Dept. of Information and Computer Science, Univ. of California–Irvine, 1997.
[11] H. Narazaki, M. Yamamoto, and T. Watanabe, “Reorganizing Knowledge in Neural Networks: An Explanation Mechanism for Neural Networks in Data Classification Problems,” IEEE Trans. Systems, Man, and Cybernetics, part B, vol. 26, no. 1, pp. 107-117, 1996.
[12] Y.-H. Pao,Adaptive Pattern Recognition and Neural Networks. Reading, Mass: Addison-Wesley, 1989.
[13] J.R. Quinlan, C4.5: Programs for Machine Learning,San Mateo, Calif.: Morgan Kaufman, 1992.
[14] S. Ridella, G. Speroni, P. Trebina, and R. Zunino, “Pruning and Rule Extraction Using Class Entropy,” Proc. IEEE Int'l Conf. Neural Networks, vol. 1, pp. 250-256, 1993.
[15] S. Sestito and T.S. Dillon, Automated Knowledge Acquisition, Prentice Hall, 1994.
[16] R. Setiono and H. Liu, “Understanding Neural Networks via Rule Extraction,” Proc. Int'l Joint Conf. Artificial Intelligence, vol. 1, pp. 480-485, 1995.
[17] K.Y. Tam and M.L. Kiang, “Managerial Applications of Neural Networks: The Case of Bank Failure Predictions,” Management Science, vol. 38, pp. 926-947, 1992.
[18] Z. Tang, C. de Almeida, and P. Fishwick, “Time Series Forecasting Using Neural Networks vs. Box-Jenkins Methodology,” Proc. First Workshop Neural Networks: Academic/Industrial/NASA/Defense, pp. 95-100, 1990.
[19] S.B. Thrun, “Extracting Provably Correct Rules from Artificial Neural Networks,” Technical Report IAI-TR-93-5, Inst. for Informatik III, Universitat Bonn, Germany, 1994.
[20] G.G. Towell and J.W. Shavlik, "The Extraction of Refined Rules from Knowledge-Based Neural Networks," Machine Learning, vol. 13, no. 1, pp. 71-101, 1993.

Index Terms:
Classification, neural network, rule extraction.
Amit Gupta, Sang Park, Siuwa M. Lam, "Generalized Analytic Rule Extraction for Feedforward Neural Networks," IEEE Transactions on Knowledge and Data Engineering, vol. 11, no. 6, pp. 985-992, Nov.-Dec. 1999, doi:10.1109/69.824621
Usage of this product signifies your acceptance of the Terms of Use.