This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Symbolic vs. Connectionist Learning: An Experimental Comparison in a Structured Domain
March/April 2001 (vol. 13 no. 2)
pp. 176-195

Abstract—During the last two decades, the attempts of finding effective and efficient solutions to the problem of learning any kind of structured information have been splitting the scientific community. A “holy war” has been fought between the advocates of a symbolic approach to learning and the advocates of a connectionist approach. One of the most repeated claims of the symbolic party has been that symbolic methods are able to cope with structured information while connectionist ones are not. However, in the last few years, the possibility of employing connectionist methods for structured data has been widely investigated and several approaches have been proposed. Does this mean that the connectionist partisans are about to win the ultimate battle? Is connectionism the “One True Approach” to knowledge learning? The paper discusses this topic and gives an experimental answer to these questions. In details, first, a novel algorithm for learning structured descriptions, ascribable to the category of symbolic techniques, is proposed. It faces the problem directly in the space of graphs by defining the proper inference operators, as graph generalization and graph specialization, and obtains general and consistent prototypes with a low computational cost with respect to other symbolic learning systems. Successively, the proposed algorithm is compared with a recent connectionist method for learning structured data [17] with reference to a problem of handwritten character recognition from a standard database publicly available on the Web. Finally, after a discussion highlighting pros and cons of symbolic and connectionist approaches, some conclusions, quantitatively supported by the experimental data, are drawn. The orthogonality of the two approaches strongly suggests their combination in a multiclassifier system so as to retain the strengths of both of them, while overcoming their weaknesses. The results on the experimental case-study demonstrated that the adoption of a parallel combination scheme of the two algorithms could improve the recognition performance of about 10 percent. A truce or an alliance between the symbolic and the connectionist worlds?

[1] T. Pavlidis, Structural Pattern Recognition. New York: Springer, 1977.
[2] J. Rocha and T. Pavlidis, "A Shape Analysis Model With Applications to a Character Recognition System," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, pp. 393-404, 1994.
[3] H. Nishida, "Shape Recognition by Integrating Structural Descriptions and Geometrical/Statistical Transforms," Computer Vision and Image Understanding, in press.
[4] B.T. Messmer and H. Bunke, “A New Algorithm for Error-Tolerant Subgraph Isomorphism Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 5, pp. 493-504, May 1998.
[5] L.P. Cordella, P. Foggia, R. Genna, and M. Vento, “Prototyping Structural Descriptions: An Inductive Learning Approach,” Advances in Pattern Recognition, Lecture Notes in Computer Science, no. 1451, pp. 339–348, 1998.
[6] R.S. Michalski, “Pattern Recognition as Rule-Guided Inductive Inference,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 2, no. 4, pp. 349–361, July 1980.
[7] R.S. Michalski, “A Theory and Methodology of Inductive Learning,” Machine Learning: An Artificial Intelligence Approach, R.S. Michalski, J.S. Carbonell, and T.M. Mitchell, eds., vol. 1, chapter 4, pp. 83–133, 1983.
[8] T.G. Dietterich, R.S. Michalski, “A Comparative Review of Selected Methods for Learning from Examples,” Machine Learning: An Artificial Intelligence Approach, R.S. Michalski, J.S. Carbonell, and T.M. Mitchell, eds., vol. 1, chapter 3, pp. 41–82, 1983.
[9] S. Muggleton, “Inductive Logic Programming,” New Generation Computing, vol. 8, no. 4, pp. 295–318, 1991.
[10] A. Pearce, T. Caelly, and W.F. Bischof, “Rulegraphs for Graph Matching in Pattern Recognition,” Pattern Recognition, vol. 27, no. 9, pp. 1231–1247, 1994.
[11] A. Chianese, L.P. Cordella, M. De Santo, and M. Vento, “ Decomposition of Ribbon-Like Shapes,” Proc. Sixth Scandinavian Conf. Image Analysis, pp. 416–423, 1989.
[12] G.E. Hinton, “Mapping Part-Whole Hierarchies into Connectionist Networks,” Artificial Intelligence, vol. 46, pp. 47–75, 1990.
[13] T.A. Plate, “Holographic Reduced Representations,” IEEE Trans. Neural Networks, vol. 6, no. 3, pp. 623–641, 1995.
[14] P. Frasconi, M. Gori, M. Maggini, and G. Soda, “Unified Integration of Explicit Knowledge and Learning by Example in Recurrent Networks,” IEEE Trans. Knowledge and Data Eng., vol. 7, no. 2, pp. 340–346, Apr. 1995.
[15] A. Sperduti, “Stability Properties of Labeling Recursive Auto-Associative Memory,” IEEE Trans. Neural Networks, vol. 6, pp. 1452-1460, 1995.
[16] A. Sperduti and A. Starita, “Supervised Neural Networks for the Classification of Structures,” IEEE Trans. Neural Networks, vol. 8, no. 3, pp. 714–735, 1997.
[17] P. Frasconi, M. Gori, and A. Sperduti, “A General Framework for Adaptive Processing of Data Structures,” IEEE Trans. Neural Networks, vol. 9, no. 5, pp. 768–786, 1998.
[18] J.B. Pollack, “Recursive Distributed Representations,” Artificial Intelligence, vol. 46, nos. 1-2, pp. 77–106, 1990.
[19] P. Smolensky, “Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems,” Artificial Intelligence, vol. 46, pp. 159–216, 1990.
[20] D.S. Touretzky, “Dynamic Symbol Structures in a Connectionist Network,” Artificial Intelligence, vol. 42, no. 3, pp. 5–46, 1990.
[21] M. Botta, A. Giordana, and L. Saitta, "Learning Fuzzy Concept Definitions," Proc. Second IEEE Int'l Conf. Fuzzy Systems,San Francisco, Calif., pp. 18-22, 1993.
[22] J.R. Quinlan, “Learning Logical Definitions from Relations,” Machine Learning, vol. 5, pp. 239–266, 1990.
[23] N. Lavrac and S. Dzeroski, Inductive Logic Programming: Techniques and Applications, Ellis Horwood, London, 1994.
[24] T.M. Mitchell, “Generalization as Search,” Artificial Intelligence, vol. 18, no. 2, pp. 203–226, 1982.
[25] T.M. Mitchell, “The Need for Biases in Learning Generalizations,” Technical Report CMB-TR-117, Dept. of Computer Science, Rutgers Univ., New Brunswick, New Jersey, 1980.
[26] Readings in Machine Learning, J. Shavlik and T. Dietterich, eds., Morgan Kaufmann, 1990.
[27] S.J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall, Upper Saddle River, N.J., 1994.
[28] P.H. Winston, “Learning Structural Descriptions from Examples,” Technical Report MAC-TR-76, Dept. of Electrical Engineering and Computer Science, MIT, 1970.
[29] L.P. Cordella, P. Foggia, C. Sansone, and M. Vento, “Subgraph Transformations for the Inexact Matching of Attributed Relational Graphs,” Computing, vol. 12, pp. 43–52, 1998.
[30] J.R. Ullman, “An Algorithm for Subgraph Isomorphism,” J. ACM, vol. 23, no. 1, Jan. 1976.
[31] M. Bongard, Pattern Recognition. Hayden Book Company (Spartan Books), 1970.
[32] L.P. Cordella, P. Foggia, C. Sansone, and M. Vento, “Performance Evaluation of the VF Graph Matching Algorithm,” Proc. 10th Int'l Conf. Image Analysis Processing, 1999.
[33] M. Minsky, "Logical Versus Analogical or Symbolic Versus Connectionist or Neat Versus Scruffy," AI Magazine, Vol. 65, No. 2, 1991, pp. 34-51.
[34] I. Bloch, “Information Combination Operators for Data Fusion: A Comparative Review with Classification,” IEEE Trans. Systems, Man, and Cybernetics—Part A, vol. 26, no. 1, pp. 52–76, 1996.
[35] T.K. Ho, J.J. Hull, and S.N. Srihari, “Decision Combination in Multiple Classifiers Systems,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 1, pp. 66-75, Jan. 1994.
[36] L.P. Cordella, C. Sansone, F. Tortorella, M. Vento, and C. De Stefano, “Neural Network Classification Reliability: Problems and Applications,” invited contribution in Image Processing and Pattern Recognition, Neural Network Systems Techniques and Applications, vol. 5, pp. 161–200, 1998.
[37] N. Lavrac and S. Dzeroski, Inductive Logic Programming: Techniques and Applications, Ellis Horwood, London, 1994.

Index Terms:
Symbolic learning, connectionist systems, structural description, attributed relational graph, prototype learning, machine learning.
Citation:
Pasquale Foggia, Roberto Genna, Mario Vento, "Symbolic vs. Connectionist Learning: An Experimental Comparison in a Structured Domain," IEEE Transactions on Knowledge and Data Engineering, vol. 13, no. 2, pp. 176-195, March-April 2001, doi:10.1109/69.917559
Usage of this product signifies your acceptance of the Terms of Use.