This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Learning Concept Descriptions with Typed Evolutionary Programming
December 2005 (vol. 17 no. 12)
pp. 1664-1677
Examples and concepts in traditional concept learning tasks are represented with the attribute-value language. While enabling efficient implementations, we argue that such propositional representation is inadequate when data is rich in structure. This paper describes STEPS, a strongly-typed evolutionary programming system designed to induce concepts from structured data. STEPS' higher-order logic representation language enhances expressiveness, while the use of evolutionary computation dampens the effects of the corresponding explosion of the search space. Results on the PTE2 challenge, a major real-world knowledge discovery application from the molecular biology domain, demonstrate promise.

[1] T. Mitchell, “Does Machine Learning Really Work,” Artificial Intelligence Magazine, vol. 18, no. 3, pp. 11-20, 1997.
[2] J.W. Lloyd, “Programming in an Integrated Functional and Logic Language,” J. Functional and Logic Programming, vol. 1999, no. 3, 1999.
[3] A.F. Bowers, C. Giraud-Carrier, J.W.L.C.J. Kennedy, and R. Mackinney-Romero, “A Framework for Higher-Order Inductive Machine Learning,” Proc. CompulogNet Area Meeting on “Computational Logic and Machine Learning,” pp. 19-25, 1997.
[4] P.A. Flach, C. Giraud-Carrier, and J.W. Lloyd, “Strongly Typed Inductive Concept Learning,” Proc. Eighth Int'l Conf. Inductive Logic Programming, vol. 1,446, pp. 185-194, 1998, http://www. springer.de/cgi-binbag_generate.pl?ISBN=3-540-64738-4 .
[5] D. Montana, “Strongly Typed Genetic Programming,” Evolutionary Computation, vol. 3, no. 2, pp. 199-230, 1995.
[6] J. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. Cambridge, Mass.: The MIT Press, 1992.
[7] C. Clack and T. Yu, “Performance Enhanced Genetic Programming,” Proc. Sixth Int'l Conf. Evolutionary Programming, pp. 87-100, 1997.
[8] T. Yu and C. Clack, “PolyGP: A Polymorphic Genetic Programming System in Haskell,” Genetic Programming 1998: Proc. Third Ann. Conf., pp. 416-421, July 1998, http://www.cs.ucl.ac.uk/staff/t.yupgp.new.ps .
[9] T. Yu and C. Clack, “Recursion, Lambda Abstractions and Genetic Programming,” Genetic Programming 1998: Proc. Third Ann. Conf., July 1998, http://www.cs.ucl.ac.uk/staff/t.yuRecursion.ps .
[10] C. Kennedy, “Evolutionary Higher-Order Concept Learning,” Late Breaking Papers of the Genetic Programming 1998 Conf., Stanford Univ. Bookstore, July 1998.
[11] C. Kennedy and C. Giraud-Carrier, “An Evolutionary Approach to Concept Learning with Structured Data,” Proc. Fourth Int'l Conf. Artificial Neural Networks and Genetic Algorithms, pp. 331-336, 1999.
[12] J. Lloyd, Logic for Learning: Learning Comprehensible Theories from Structured Data. New York: Springer, 2003.
[13] M. Bongard, Pattern Recognition. Spartan Books, 1970.
[14] A. Giordana and F. Neri, “Search-Intensive Concept Induction,” Evolutionary Computation J., vol. 3, no. 4, pp. 375-416, 1996.
[15] C. Kennedy, “Strongly Typed Evolutionary Programming,” PhD dissertation, Univ. of Bristol, 2000.
[16] K.A. DeJong, W.M. Spears, and D.F. Gordon, “A Knowledge-Intensive Genetic Algorithm for Supervised Learning Using Genetic Algorithms for Concept Learning,” Machine Learning, vol. 13, pp. 161-188, 1993.
[17] C.Z. Janikow, “A Knowledge-Intensive Genetic Algorithm for Supervised Learning,” Machine Learning, vol. 13, pp. 189-228, 1993.
[18] J. Hekanaho, “DOGMA: A GA-Based Relational Learner,” Proc. Eighth Int'l Conf. Inductive Logic Programming (ILP-98), vol. 1446, pp. 205-214, July 1998.
[19] A. Giordana and L. Saitta, “REGAL: An Integrated System for Learning Relations Using Genetic Algorithms,” Proc. Second Int'l Workshop Multi Strategy Learning, pp. 234-249, 1993.
[20] L. Davis, “Adapting Operator Probabilities in Genetic Algorithms,” Proc. Third Int'l Conf. Genetic Algorithms, pp. 61-69, June 1989.
[21] J. Rosca and D. Ballard, “Causality in Genetic Programming,” Proc. Sixth Int'l Conf. Genetic Algorithms, pp. 256-263, 1995.
[22] M. Srinivas and L. Patnaik, “Adaptive Probabilities of Crossover and Mutation in Genetic Algorithms,” IEEE Trans. Systems, Man, and Cybernetics, vol. 24, no. 4, pp. 656-667, 1994.
[23] B. Julstrom, “What Have You Done for Me Lately? Adapting Operator Probabilities in a Steady-State Genetic Algorithm,” Proc. Sixth Int'l Conf. Genetic Algorithms, pp. 81-87, 1995.
[24] N. Nilsson, Machine Learning Principles of Artificial Intelligence. Tioga, 1980.
[25] R. Michalski, “A Theory and Methodology of Inductive Learning,” Artificial Intelligence, vol. 20, pp. 111-161, 1983.
[26] P.J. Angeline, “Genetic Programming and Emergent Intelligence,” Advances in Genetic Programming, K.E. Kinnear Jr., ed., chapter 4, pp. 75-98, Cambridge, Mass: MIT Press, 1994, http://www. natural-selection.com/people/ pja/docsaigp.ps.Z.
[27] T. Blickle and L. Thiele, “Genetic Programming and Redundancy,” Genetic Algorithms within the Framework of Evolutionary Computation (Workshop at KI-94, Saarbrücken), J. Hopf, ed., pp. 33-38, Max-Planck-Institut für Informatik (MPI-I-94-241), 1994, http://www.tik.ee.ethz.ch/blickleGPandRedundancy.ps.gz .
[28] P.J. Angeline, “Two Self-Adaptive Crossover Operators for Genetic Programming,” Advances in Genetic Programming 2, P. Angeline and K. Kinnear Jr., eds., chapter 5, pp. 89-110, Cambridge, Mass: MIT Press, 1996, http://www.natural-selection.com/people/ pja/docsaigp2.ps.Z.
[29] W. Tackett, “Recombination, Selection, and the Genetic Construction of Computer Programs,” PhD dissertation, Dept. of Electrical Eng. Systems, Univ. of Southern California, 1994, ftp://ftp.mad-scientist.com/pub/genetic-programming/ paperswatphd.tar.Z.
[30] W.B. Langdon and R. Poli, “Fitness Causes Bloat,” Soft Computing in Eng. Design and Manufacturing, P.K. Chawdhry, R. Roy, and R.K. Pant, eds., pp. 13-22, London: Springer-Verlag, June 1997.
[31] W.B. Langdon and R. Poli, “Fitness Causes Bloat: Mutation,” Proc. First European Workshop Genetic Programming, pp. 37-48, Apr. 1998, ftp://ftp.cwi.nl/pub/W.B.Langdon/papersWBL.euro98_ bloatm.ps.gz .
[32] W. Langdon, T. Soule, R. Poli, and J. Foster, “The Evolution of Size and Shape,” Advances in Genetic Programming 3, L. Spector, W. Langdon, U.-M. O'Rielly, and P.J. Angeline, eds., chapter 8, pp. 163-190, Cambridge, Mass.: MIT Press, June 1999.
[33] T. Soule, J. Foster, and J. Dickinson, “Code Growth in Genetic Programming,” Genetic Programming 1996: Proc. First Ann. Conf., pp. 215-223, July 1996.
[34] T. Blickle, “Evolving Compact Solutions in Genetic Programming: A Case Study,” Parallel Problem Solving from Nature IV. Proc. Int'l Conf. Evolutionary Computation, H.-M. Voigt, W. Ebeling, I. Rechenberg, and H.-P. Schwefel, eds., pp. 564-573, Sept. 1996, http://www.tik.ee.ethz.ch/ blickleppsn1.ps.gz .
[35] B.-T. Zhang and H. Mühlenbein, “Adaptive Fitness Functions for Dynamic Growing/Pruning of Program Trees,” Advances in Genetic Programming 2, P.J. Angeline and K.E. Kinnear, Jr., eds., pp. 241-256, Cambridge, Mass.: MIT Press, 1996.
[36] C. Kennedy and C. Giraud-Carrier, “A Depth Controlling Strategy for Strongly Typed Evolutionary Programming,” Proc. Genetic and Evolutionary Computation Conf., vol. 1, pp. 879-885, July 1999.
[37] C. Gathercole, “An Investigation of Supervised Learning in Genetic Programming,” PhD dissertation, Univ. of Edinburgh, 1998, ftp://ftp.dai.ed.ac.uk/pub/daidb/paperspt9810.ps.gz .
[38] R. Michalski and J. Larson, “Inductive Inference of VL Decision Rules,” Proc. Workshop Pattern-Directed Inference Systems, pp. 34-44, 1977.
[39] S. Muggleton and C. Page, “Beyond First-Order Learning: Inductive Learning with Higher Order Logic,” Technical Report PRG-TR-13-94, Oxford Univ. Computing Laboratory, 1994.
[40] A. Srinivasan, R. King, S. Muggleton, and M. Sternberg, “The Predictive Toxicology Evaluation Challenge,” Proc. 15th Int'l Joint Conf. Artificial Intelligence, 1997, ftp://ftp.cs.york.ac.uk/pub/ML_GROUP/Papers ijcai97.ps.gz.
[41] J. van Hemert and A. Eiben, “Comparison of the SAW-ing Evolutionary Algorithm and the Grouping Genetic Algorithm for Graph Coloring,” Technical Report TR-97-14, Leiden Univ., 1997.
[42] Y. Freund and R. Schapire, “Experiments with a New Boosting Algorithm,” Proc. 13th Int'l Conf. Machine Learning, pp. 148-146, 1996.
[43] D. Bristol, personal communication, 1999.
[44] http://web.comlab.ox.ac.uk/oucl/research/ areas/machlearn/PTEpte2-summary.html/, 1999.
[45] C. Kennedy, C. Giraud-Carrier, and D. Bristol, “Predicting Carcinogenesis Using Structural Information Only,” Proc. Third European Conf. Priciples of Data Mining and Knowledge Discovery, pp. 360-365, 1999.
[46] H. Bensusan, C. Giraud-Carrier, and C. Kennedy, “A Higher-Order Approach to Meta-Learning,” Proc. ECML-2000 Workshop Meta-Learning: Building Automatic Advice Strategies for Model Selection and Method Combination, pp. 109-118, 2000.

Index Terms:
Index Terms- Concept learning, typed evolutionary programming.
Citation:
Claire J. Thie, Christophe Giraud-Carrier, "Learning Concept Descriptions with Typed Evolutionary Programming," IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 12, pp. 1664-1677, Dec. 2005, doi:10.1109/TKDE.2005.199
Usage of this product signifies your acceptance of the Terms of Use.