This Article 
 Bibliographic References 
 Add to: 
Exploring the Power of Genetic Search in Learning Symbolic Classifiers
November 1996 (vol. 18 no. 11)
pp. 1135-1141

Abstract—In this paper we show, in a constructive way, that there are problems for which the use of genetic algorithm based learning systems can be at least as effective as traditional symbolic or connectionist approaches. To this aim, the system REGAL* is briefly described, and its application to two classical benchmarks for Machine Learning is discussed, by comparing the results with the best ones published in the literature.

[1] C.A. Ankenbrandt, B.P. Buckles, and F.E. Petry, "Scene Recognition Using Genetic Algorithms with Semantic Nets," Pattern Recognition Letters, vol. 11, pp. 285-293, 1990.
[2] J. Bala, K.A. De Jong, and P. Pachowicz, "Learning Noise Tolerant Classification Procedures by Integrating Inductive Learning and Genetic Algorithms" Proc. First Int'l Workshop on Multistrategy Learning,Harpers Ferry, W. Va., pp. 316-323, 1991.
[3] P. Bonelli, A. Parodi, S. Sen, and S. Wilson, "NEWBOOLE: A Fast GBML System," Proc. Int'l Conf. Machine Learning,Austin, Tex., pp. 153-159, 1990.
[4] R. Boswell, "Manual for NewID Version 2.1," The Turing Institute, Glasgow, U.K. 1990.
[5] M. Botta and A. Giordana, "Smart+: A Multi Strategy Learning Tool," Proc. 13th Int'l Joint Conf. Artificial Intelligence,Chambéry, France, pp. 937-943. 1993.
[6] M. Botta, A. Giordana, and L. Saitta, "Learning Fuzzy Concept Definitions," Proc. Second IEEE Int'l Conf. Fuzzy Systems,San Francisco, Calif., pp. 18-22, 1993.
[7] L. Breiman, J. Friedman, R. Olshen, and C. Stone, Classification and Regression Trees.Monterey, Calif.: Wadsworth, 1984.
[8] P. Brèzellec and H. Soldano, "ÉLÉNA: A Bottom-Up Learning Method," Proc. 10th Machine Learning Conf.,Amherst, Mass., pp. 9-16, 1993.
[9] F.Z. Brill, D.E. Brown, and W.N. Martin, Fast Genetic Selection of Features for Neural Network Classifiers IEEE Trans. Neural Networks, vol. 3, no. 2, pp. 324-328, Mar. 1992.
[10] P. Clark and R. Boswell,“Rule induction with CN2: Some recent improvements,” Machine Learning—EWSL-91, Y. Kodratoff, ed., pp. 151-163,Berlin: Springer Verlag, 1991.
[11] J.P. Cohoon, S.U. Hedge, W.N. Martin, and D. Richards, "Punctuated Equilibria: A Parallel Genetic Algorithm," Proc. Second Int'l Conf. Genetic Algorithms,Cambridge, Mass., pp. 153-159, 1987.
[12] S. Cost and S. Salzberg, "A Weighted Nearest Neighbor Algorithm for Learning with Symbolic Features," Machine Learning, Vol. 10, No. 1, Jan. 1993, pp. 57-78.
[13] T.M. Cover and P. Hart, "Nearest Neighbor Pattern Classification," Proc. IEEE Trans. Information Theory, pp. 21-27, 1967.
[14] K. Deb and D. Goldberg, "An Investigation of Niches and Species Formation in Genetic Function Optimization," Proc. Third Int'l Conf. on Genetic Algorithms,Fairfax, Va., pp. 42-50, 1989.
[15] K.A. De Jong, "Analysis of the Behaviour of a Class of Genetic Adaptive Systems," Doctoral Dissertation, Dept. of Computer and Comm. Sciences, Univ. of Michigan, Ann Arbor, Mich., 1975.
[16] K.A. DeJong, W.M. Spears, and D.F. Gordon, “Using Genetic Algorithms for Concept Learning,” Machine Learning, Vol. 13,No. 2 and 3, 1993, pp.161–188.
[17] D.H. Fisher, “Knowledge Acquisition via Incremental Conceptual Clustering,” Machine Learning, no. 2, pp. 139-172, 1987.
[18] A. Giordana and C. Sale, "Genetic Algorithms for Learning Relations," Proc. Ninth Int'l Conf. Machine Learning, pp. 169-178,Aberdeen, U.K., 1992.
[19] A. Giordana and L. Saitta, "REGAL: An Integrated System for Learning Relations Using Genetic Algorithms," Proc. Second Int'l Workshop on Multistrategy Learning,Harpers Ferry, W. Va., pp. 234-249, 1993.
[20] A. Giordana and L. Saitta, "Learning Disjunctive Concepts by Means of Genetic Algorithms," Proc. 11th Int'l Conf. Machine Learning,New Brunswick, N.J., pp. 96-104, 1994.
[21] A. Giordana, G. Lo Bello, and L. Saitta, "Abstraction in Propositional Calculus," Proc. Workshop Knowledge Compilation and Speed Up Learning,Amherst, Mass., pp. 56-64, 1994.
[22] A. Giordana and F. Neri, "Search-Intensive Concept Learning," Evolutionary Computation, MIT Press, vol. 3, no. 4, pp. 375-496, 1996.
[23] D.E. Goldberg and J. Richardson, “Genetic Algorithms with Sharing for Multimodal Function Optimization,” Proc. Second Int'l Conf. Genetic Algorithms, pp. 41-49, 1987.
[24] D.E. Goldberg, Genetic Algorithms. Addison-Wesley, 1989.
[25] D.P. Greene and S.F. Smith, "Competition-Based Induction of Decision Models from Examples," Machine Learning, vol. 13, pp. 229-258, 1993.
[26] J. Hekanaho, "Symbiosis in Multimodal Concept Learning," Proc. 12th Int'l Conf. Machine Learning,Lake Tahoe, Calif:, Morgan Kaufmann, pp. 278-285, 1995.
[27] J.H. Holland, "Adaptation in Natural and Artificial Systems," PhD Thesis, Univ. of Michigan, Ann Arbor, Mich.
[28] J.H. Holland, "Escaping Brittleness: The Possibilities of General Purpose Learning Algorithms Applied to Parallel Rule-Based Systems," R. Michalski, J. Carbonell, and T. Mitchell, eds., Machine Learning: An AI Approach, vol. II. Los Altos, Calif.: Morgan Kaufmann, pp. 593-623, 1986.
[29] R.C. Holte, “Very Simple Classification Rules Perform Well on Most Commonly Used Datasets,” Machine Learning, vol. 11, pp. 63–91, 1993.
[30] J. Horn, K. Deb, and D.E. Goldberg, "Implicit Niching in a Learning Classifier System: Nature's Way," Evolutionary Computation, vol. 2, pp. 37-66, 1994.
[31] C.A. Janikow, “A Knowledge-Intensive Genetic Algorithm for Concept Learning,” Machine Learning, Vol. 13,No. 2 and 3, 1993, pp.189–228.
[32] M. Koppel, A.M. Segre, and R. Feldman, "Getting the Most from Flawed Theories," Proc. 11th Int'l Conf. Machine Learning,New Brunswick, N.J., pp. 139-147, 1994.
[33] B. Lewin, Gene V.New York: Oxford Univ. Press, 1994.
[34] S.W. Mahfoud, "Niching Methods for Genetic Algorithms," PhD Thesis, Univ. of Illi nois, Urbana-Champaign, Ill. 1995.
[35] R.A. McCallum and K.A. Spackman., "Using Genetic Algorithm to Learn Disjunctive Rules from Examples," Proc. Seventh Int'l Conf. on Machine Learning,Austin, Tex., pp. 149-152, 1990.
[36] R. Michalski, "A Theory and Methodology of Inductive Learning," R. Michalski, J. Carbonell, and T. Mitchell, eds., Machine Learning: An AI Approach, vol. I. Morgan Kaufmann, Los Altos, Calif., pp. 83-134. 1983.
[37] R. Michalski, I. Mozetic., J. Hong, and N. Lavrac, "The Multi-Purpose Incremental Learning System AQ15 and its Testing Application to Three Medical Domains," Proc. Fifth Nat'l Conf. Artificial Intelligence,Philadelphia, Penn., pp. 1,041-1,045, 1986.
[38] F. Neri and L. Saitta, "Analysis of Genetic Algorithms Evolution under Pure Selection." Proc.Sixth Int'l Conf. Genetic Algorithms,Pittsburgh, Penn., pp. 32-41. 1995.
[39] F. Neri and A. Giordana, "A Parallel Genetic Algorithm for Concept Learning," Proc. Sixth Int'l Conf. Genetic Algorithms,Pittsburgh, Penn., pp. 436-443, 1995.
[40] M. Noordewier, G. Towell, and J. Shavlik, "Training Knowledge-Based Neural Networks to Recognize Genes in DNA Sequences," NIPS, vol. 3,Denver, Col., 1991.
[41] S.W. Norton and H. Hirsh, "Learning DNF via Probabilistic Evidence Combination," Proc. 10th Int'l Conf. Machine Learning,Amherst, Mass., pp. 220-227, 1993.
[42] D.W. Opitz and J.W. Shavlik, "Using Genetic Search to Refine Knowledge-Based Neural Networks," Proc. 11th Int'l Conf. Machine Learning,New Brunswick, N.J., pp. 208-216.
[43] J.R. Quinlan,"Induction of decision trees," Machine Learning, vol. 1, pp. 81-106, 1986.
[44] J.R. Quinlan, “Learning Logical Definitions from Relations,” Machine Learning, vol. 5, pp. 239–266, 1990.
[45] J. Rachlin, S. Kasif, S. Salzberg, and D.W. Aha, "Towards a Better Understanding of Memory-Based Reasoning Systems," Proc. 11th Conf. Machine Learning,New Brunswick, N.J., pp. 242-250, 1994.
[46] F. Rosenblatt, "Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms,"New York: Spartan, 1962.
[47] D.EE. Rumelhart and J.L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, MIT Press, Cambridge, Mass., 1986.
[48] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning Internal Representations by Error Propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1: Foundations, D.E. Rumelhart and J.L. McClelland et al., eds., chapter 8, pp. 318-362.Cambridge, Mass.: MIT Press, 1986.
[49] J.S. Schlimmer, "Concept Acquisition through Representational Adjustment," TR 87-19, Dept. of Information and Computer Science, Univ. of California, Irvine, 1987.
[50] W. Siedlecki and J. Sklanski, "A Note on Genetic Algorithms for Large Scale Feature Selection," Pattern Recognition Letters, vol. 10, pp. 335-347, 1989.
[51] S. Smith, "Flexible Learning of Problem Solving Heuristics through Adaptive Search," Proc. Eighth Int'l Joint Conf. Artificial Intelligence,Karlsruhe, Germany, pp. 422-425, 1983.
[52] W.M. Spears, "Simple Subpopulation Schemes," Proc. Conf. Evolutionary Programming,.San Diego, Calif., pp. 296-307, 1994.
[53] R.S. Sutton, "Learning to Predict by the Methods of Temporal Differences," Machine Learning, vol. 3, pp. 9-44, 1988.
[54] G. Syswerda,"Uniform crossover in genetic algorithms," Proc. Third Int'l Conf. Genetic Algorithms, pp. 2-9, 1989.
[55] G. Towell and J. Shavlik, "Refining Symbolic Knowledge Using Neural Networks," Proc. First Workshop Multistrategy Learning,Harpers Ferry, W. Va., pp. 257-272, 1991.
[56] G.G. Towell and J.W. Shavlik, "Knowledge-Based Artificial Neural Networks," Artificial Intelligence, vol. 70, nos. 1-2, pp. 119-165, 1994.
[57] H. Vafaie and K.A. De Jong, "Improving the Performance of Rule Induction System Using Genetic Algorithms," Proc. First Int'l Workshop on Multistrategy Learning,Harpers Ferry, Va., pp. 305-315, 1991.
[58] G. Venturini, "SIA: A Supervised Inductive Algorithm with Genetic Search for Learning Attribute Based Concepts," Proc. European Conf. Machine Learning,Vienna, Austria, pp. 280-296. 1993.
[59] S.M. Weiss and N. Indurkhya, "Small Sample Decision Tree Pruning," Proc. 11th Machine Learning Conf.,New Brunswick, N.J., pp. 335-342, 1994.
[60] D.Y. Yeung, "A Neural Network Approach to Constructive Induction," Proc. Eighth Int'l Conf. on Machine Learning,Evanston, Ill., pp. 228-232, 1991.
[61] S. Wilson, "Classifier Systems and the Animat Problem," Machine Learning, vol. 2, 199-228, 1987.

Index Terms:
Genetic algorithms, distributed genetic algorithms, classification rules, machine learning, disjunctive concept learning, universal suffrage selection, mushroom dataset, splice junctions dataset, empirical comparison.
Filippo Neri, Lorenza Saitta, "Exploring the Power of Genetic Search in Learning Symbolic Classifiers," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 11, pp. 1135-1141, Nov. 1996, doi:10.1109/34.544085
Usage of this product signifies your acceptance of the Terms of Use.