This Article 
 Bibliographic References 
 Add to: 
Enhancing Knowledge Discovery via Association-Based Evolution of Neural Logic Networks
July 2006 (vol. 18 no. 7)
pp. 889-901
Sam Y. Sung, IEEE Computer Society
The comprehensibility aspect of rule discovery is of emerging interest in the realm of knowledge discovery in databases. Of the many cognitive and psychological factors relating the comprehensibility of knowledge, we focus on the use of human amenable concepts as a representation language in expressing classification rules. Existing work in neural logic networks (or neulonets) provides impetus for our research; its strength lies in its ability to learn and represent complex human logic in decision-making using symbolic-interpretable net rules. A novel technique is developed for neulonet learning by composing net rules using genetic programming. Coupled with a sequential covering approach for generating a list of neulonets, the straightforward extraction of human-like logic rules from each neulonet provides an alternate perspective to the greater extent of knowledge that can potentially be expressed and discovered, while the entire list of neulonets together constitute an effective classifier. We show how the sequential covering approach is analogous to association-based classification, leading to the development of an association-based neulonet classifier. Empirical study shows that associative classification integrated with the genetic construction of neulonets performs better than general association-based classifiers in terms of higher accuracies and smaller rule sets. This is due to the richness in logic expression inherent in the neulonet learning paradigm.

[1] P.J. Angeline, “Genetic Programming and Emergent Intelligence,” Advances in Genetic Programming, pp. 76-97, 1994.
[2] R. Agrawal, T. Imielinski, and A. Swami, “Mining Association Rules between Sets of Items in Large Databases,” Proc. 1993 ACM SIGMOD Int'l Conf. Management of Data, pp. 207-216, 1993.
[3] C.L. Blake and C.J. Merz, “UCI Repository of Machine Learning Databases,” technical report, Dept. of Information and Computer Science, Univ. of California, Irvine, 1998.
[4] H.W.K. Chia and C.L. Tan, “Confidence and Support Classification Using Genetically Programmed Neural Logic Networks,” Proc. Genetic and Evolutionary Computation Conf. (GECCO '04), Part II, pp. 836-837, 2004.
[5] W. Cohen, “Fast Effective Rule Induction,” Proc. 12th Int'l Conf. Machine Learning (ICML '95), pp. 115-123, 1995.
[6] U. Fayyad, G.P. Shapiro, and P. Smyth, “From Data Mining to Knowledge Discovery in Databases,” AI Magazine, vol. 17, pp. 37-54, 1996.
[7] D.H. Fisher and K.B. McKusick, “An Empirical Comparison of ID3 and Back-Propagation,” Proc. 11th Int'l Joint Conf. Artificial Intelligence, pp. 788-793, 1989.
[8] A.A. Freitas, “A Survey of Evolutionary Algorithms for Data Mining and Knowledge Discovery,” Advances in Evolutionary Computation, pp. 819-845, 2002.
[9] V.C. Gaudet, “Genetic Programming of Logic-Based Neural Networks,” Genetic Algorithms for Pattern Recognition, pp. 315-330, 1996.
[10] G. Gigerenzer, P.M. Todd,and the ABC Research Group, Simple Heuristics that Make Us Smart. Oxford Univ. Press, 1999.
[11] J.K. Kishore, L.M. Patnaik, V. Mani, and V.K. Agrawal, “Application of Genetic Programming for Multicategory Pattern Classification,” IEEE Trans. Evolutionary Computation, vol. 4, no. 3, pp. 242-258, 2000.
[12] S.C. Kleene, Introduction to MetaMathematics. van Nostrand, Princeton, 1952.
[13] R. Kohavi, D. Sommerfield, and J. Dougherty, “Data Mining Using MLC++: A Machine Learning Library in C++,” Int'l J. Artificial Intelligence Tools, vol. 6, no. 4, pp. 537-566, 1996.
[14] J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, 1992.
[15] J.R. Levenick, “Inserting Introns Improves Genetic Algorithm Success Rate: Taking a Cue from Biology,” Proc. Fourth Int'l Conf. Genetic Algorithms, pp. 123-127, 1991.
[16] W. Li, J. Han, and J. Pei, “CMAR: Accurate and Efficient Classification Based on Multiple-Class Association Rules,” Proc. Int'l Conf. Data Mining (ICDM '01), pp. 369-376, 2001.
[17] B. Liu, W. Hsu, and Y. Ma, “Integrating Classification and Association Rule Mining,” Proc. Fourth Int'l Conf. Knowledge Discovery and Data Mining (KDD '98), pp. 80-86, 1998.
[18] B. Liu, Y. Ma, and C.K. Wong, “Improving an Association Rule Based Classifier,” Principles of Data Mining and Knowledge Discovery, pp. 80-86, 1998.
[19] R.S. Michalski, I. Mozetic, J. Hong, and N. Lavrac, “The Multipurpose Incremental Learning System AQ15 and Its Testing Application to Three Medical Domains,” Proc. Nat'l Conf. Artificial Intelligence, pp. 1041-1045, 1986.
[20] G. Miller, “The Magical Number Seven Plus or Minus Two: Some Limits on Our Capacity for Processing Information,” Psychological Rev., vol. 101, pp. 343-352, 1994.
[21] P.M. Murphy and M.J. Pazzani, “ID2-of-3: Constructive Induction of M-of-N Concepts for Discriminators in Decision Trees,” Proc. Eighth Int'l Workshop Machine Learning, pp. 183-187, 1991.
[22] A. Newell and H.A. Simon, Human Problem Solving. Prentice-Hall, 1972.
[23] T. Niwa and H. Iba, “Distributed Genetic Programming: Empirical Study and Analysis,” Genetic Programming: Proc. First Ann. Conf., pp. 339-344, 1996.
[24] M.J. Pazzani, “Knowledge Discovery from Data?” IEEE Intelligent Systems, vol. 15, no. 2, pp. 10-13, 2000.
[25] J.R. Quinlan, C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
[26] R.L. Rivest, “Learning Decision Lists,” Machine Learning, vol. 2, no. 3, pp. 229-246, 1987.
[27] J Schlimmer, “Congressional Quarterly Almanac,” 98th Congress, second Session 1984, vol. XI, Congressional Quarterly Inc., 1987.
[28] C.L. Tan, T.S. Quah, and H.H. Teh, “An Artificial Neural Network that Models Human Decision Making,” Computer, vol. 29, no. 3, pp. 64-70, 1996.
[29] C.L. Tan and H.W.K. Chia, “Genetic Construction of Neural Logic Networks,” Proc. INNS-IEEE Int'l Joint Conf. Neural Networks (IJCNN '01), vol. 1, pp. 732-737, 2001.
[30] C.L. Tan and H.W.K. Chia, “Neural Logic Network Learning Using Genetic Programming,” Proc. 17th Int'l Joint Conf. Artificial Intelligence (IJCAI '01), vol. 2, pp. 803-808, 2001.
[31] H.H. Teh, Neural Logic Network, A New Class of Neural Networks. Singapore: World Scientific, 1995.
[32] G.G. Towell and J.W. Shavlik, “Knowledge-Based Artificial Neural Networks,” Artificial Intelligence, vol. 70, nos. 1-2, pp. 119-165, 1994.
[33] A. Tsakonas, V. Aggelis, I. Karkazis, and G. Dounias, “An Evolutionary System for Neural Logic Networks Using Genetic Programming and Indirect Encoding,” J. Applied Logic, vol. 2, pp. 349-379, 2004.
[34] P. Utgoff and C. Brodley, “Linear Machine Decision Trees,” COINS Technical Report 91-10, Univ. of Massachusetts, Amherst, 1991.
[35] T Weijters and J. Paredis, “Rule Induction with a Genetic Sequential Covering Algorithm (GeSeCo),” Proc. Second Int'l ICSC Symp. Eng. of Intelligent Systems (EIS-2000), 2000.
[36] X. Yin and J. Han, “CPAR: Classification Based on Predictive Association Rules,” Proc. 2003 SIAM Int'l Conf. Data Mining (SDM '03), 2003.

Index Terms:
Data mining, knowledge acquisition, connectionism and neural nets, genetic programming, rule-based knowledge representation.
Henry W.K. Chia, Chew Lim Tan, Sam Y. Sung, "Enhancing Knowledge Discovery via Association-Based Evolution of Neural Logic Networks," IEEE Transactions on Knowledge and Data Engineering, vol. 18, no. 7, pp. 889-901, July 2006, doi:10.1109/TKDE.2006.111
Usage of this product signifies your acceptance of the Terms of Use.