This Article 
 Bibliographic References 
 Add to: 
Conundrum of Combinatorial Complexity
June 1998 (vol. 20 no. 6)
pp. 666-670

Abstract—This paper examines fundamental problems underlying difficulties encountered by pattern recognition algorithms, neural networks, and rule systems. These problems are manifested as combinatorial complexity of algorithms, of their computational or training requirements. The paper relates particular types of complexity problems to the roles of a priori knowledge and adaptive learning. Paradigms based on adaptive learning lead to the complexity of training procedures, while nonadaptive rule-based paradigms lead to complexity of rule systems. Model-based approaches to combining adaptivity with a priori knowledge lead to computational complexity. Arguments are presented for the Aristotelian logic being culpable for the difficulty of combining adaptivity and a priority. The potential role of the fuzzy logic in overcoming current difficulties is discussed. Current mathematical difficulties are related to philosophical debates of the past.

[1] Aristotle, Metaphysics. Translated H.G. Apostle, Bloomington, Ind: Indiana Univ. Press, 1966, IV BC.
[2] Avicenna, Kitab al-Shifa. Translated in Avicenna, S.M. Afnan, London, Great Britain: George Allen&Unwin, LTD, 1958, XI AD.
[3] T. Aquinas, Summa Contra Gentiles. Tr. A.C. Pegis. Univ. of Notre Dame Press, 1997, 1324.
[4] R.E. Bellman, Adaptive Control Processes.Princeton, NJ: Princeton Univ. Press, 1961.
[5] P.P. Bonnisone, M. Henrion, L.N. Kanal, and J.F. Lemmer, Uncertainty in Artificial Intelligence 6.Amsterdam, The Netherlands: NorthHolland, 1991.
[6] R.P. Botha, Challenging Chomsky. The Generative Garden Game.Oxford, UK: Basil Blackwell, 1991.
[7] R.A. Brooks, "Model-Based Three-Dimensional Interpretation of Two-Dimensional Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 5, no. 2, pp. 140-150, 1983.
[8] A. Califano and R. Mohan, "Multidimensional Indexing for Recognizing Visual Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 4, pp. 373-392, 1994.
[9] R.T. Chen and C.R. Dyer, "Model-Based Recognition in Robotic Vision," ACM Computing Surveys, vol. 18, pp. 67-108, 1986.
[10] N. Chomsky, Language and Mind.New York, NY: Harcourt Brace Java novich, 1972.
[11] N. Chomsky, Principles and Parameters in Syntactic Theory. N. Hornstein and D. Lightfoot eds., Explanation in Linguistics. The Logical Problem of Language Acquisition, London: Longman, 1981.
[12] R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis.New York, NY: J. Wiley&Sons, 1973.
[13] K. Fukunaga, Introduction to Statistical Pattern Recognition.New York, NY: Academic Press, 1972.
[14] F. Girosi, M. Jones, and T. Poggio, "Regularization Theory and Neural Networks Architectures," Neural Computation, vol. 7, no. 2, pp. 219-269, 1995.
[15] W.E.L. Grimson and T. Lozano-Perez, "Model-Based Recognition and Localization From Sparse Range or Tactile Data," Int'l J. Robotics Research, vol. 3, no. 3, pp. 3-35, 1984.
[16] W.E.L. Grimson and D.P. Huttenlocher, "Introduction to the Special Issue on Interpretation of 3-D Scenes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 10, pp. 969-970, 1991; vol. 14, no. 2, pp. 97-98, 1992.
[17] S. Grossberg, "How Does a Brain Build a Cognitive Code?" Psychological Review, vol. 87, pp. 1-51, 1980.
[18] S. Grossberg, "Nonlinear Neural Networks: Principles, Mechanisms, and Architectures," Neural Networks, vol. 1, no. 1, pp. 17-61, 1988.
[19] I. Kant, Critique of Pure Reason. Translated J.M.D. Meiklejohn, New York, NY: Wiley Book, 1943, 1781.
[20] H.R. Keshavan, J. Barnett, D. Geiger, and T. Verma, "Introduction to the Special Section on Probabilisitc Reasoning," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 3, pp. 193-195, 1993.
[21] J. Koster and R. May, Levels of Syntactic Representation.Dordrecht: Foris Publications, 1981.
[22] Y. Lamdan and H.J. Wolfson, "Geometric hashing: A general and efficient model-based recognition scheme," Second Int'l Conf. Computer Vision, pp. 238-249, 1988.
[23] M. Maimonides, The Guide for the Perplexed, 2nd edition. Transl. M. Friedlander, New York, NY: Dover, 1956, 1190.
[24] W.S. McCulloch and W. Pitts, "A Logical Calculus of the Ideas Immanent in Nervous Activity," Bull. Mathematical Biophysics, vol. 7, pp. 115-133, 1943.
[25] W.S. McCulloch, "What Is a Number that a Man May Know It, and a Man, that He May Know a Number?" Ninth Alfred Korzybski Memorial Lecture, General Semantics Bull., vol. 26, no. 27, pp. 17-18, 1961. Also in McCulloch, 1965.
[26] W.S. McCulloch, Embodiments of Mind, 2nd edition. Cambridge, Mass.: MIT Press, 1988, 1965.
[27] R.S. Michalski, J.G. Carbonell, and T.M. Mitchell, Machine Learning: An Artificial Intelligence Approach, vol 2. Los Altos, Calif.: Morgan Kaufmann, 1986.
[28] M.L. Minsky, Semantic Information Processing.Cambridge, Mass.: The MIT Press, 1968.
[29] M.L. Minsky, "A Framework for Representing Knowledge," In The Psychology of Computer Vision, P.H. Whinston, ed., New York, NY: McGraw-Hill Book, 1975.
[30] M.L. Minsky and S.A. Papert, Perceptrons.Cambridge, Mass.: The MIT Press, 1969, 1988.
[31] S. Negahdaripour and A.K. Jain, "Final Report of the NSF Workshop on the Challenges in Computer Vision Research," Future Directions of Research, U.S. Nat'l Science Foundation, 1991.
[32] R. Nevatia and T.O. Binford, "Description and Recognition of Curved Objects," Artificial Intelligence, vol. 8, no. 1, pp. 77-98, 1977.
[33] N.J. Nilsson, Learning Machines.New York, NY: McGraw-Hill, 1965.
[34] W. Occam, "Summa Logicae," Translated M.J. Loux, Occam's Theory of Terms, 1974, and Translated A.J. Freddoso and H. Schuurman, Occam's Theory of Propositions.Notre Dame, Ind.: Univ. of Notre Dame Press, 1980, XIV.
[35] L.I. Perlovsky, "Computational Concepts in ATR: Neural Networks, Statistical Pattern Recognition, and Model Based Vision," ATR Working Group Meeting,Seattle, Wash., 1991.
[36] L.I. Perlovsky, "Computational Concepts in Classification: Neural Networks, Statistical Pattern Recognition, and Model Based Vision," J. Math. Imaging and Vision, vol. 4, no. 1, pp. 81-110, 1994.
[37] L.I. Perlovsky, "Fuzzy Logic of Aristotelian Forms," Proc. Conf. Intelligent Systems and Semiotics '96, vol. 1, pp. 43-48,Gaithersburg, Md., 1996.
[38] W. Pitts and W.S. McCulloch, "How We Know Universals: The Perception of Auditory and Visual Forms," Bull. Math. Biophysics, vol. 9, pp. 127-147, 1947.
[39] Plato, Phaedrus. Translated in Plato, L. Cooper. New York, NY: Oxford Univ. Press, IV BC.
[40] A. Rosenblueth, N. Wiener, and J. Bigelow, "Behavior, Purpose and Teleology," Philosophy of Science, vol. 10, no. 1, pp. 18-24, 1943.
[41] A.M. Segre, "Applications of Machine Learning," IEEE Expert, vol. 7, no. 3, pp. 31-34, 1992.
[42] B.F. Skinner, About Behaviorism.New York, NY: Alfred A. K nopf, 1974.
[43] R. Sun and L.A. Bookman, Computational Architectures Integrating Neural and Symbolic Processing.Boston, Mass.: Kluwer Academic Publishers, 1995.
[44] S. Watanabe, Pattern Recognition: Human and Mechanical.New York, NY: John Wiley&Sons, 1985.
[45] J.B. Watson, "Psychology as the Behaviorist Views It," Psychological Rev., vol. 20, pp. 158-177, 1913.
[46] B. Widrow, "Adaptive Sample-Data System—A Statistical Theory of Adaptation," 1959 WESCON Convention Record, Part 4, pp. 74-85, 1959.
[47] N. Wiener, Cybernetics.New York, NY: Wiley, 1948.
[48] P.H. Winston, Artificial Intelligence, 2nd edition. Reading, Mass.: Addison-Wesley, 1984.
[49] L.A. Zadeh, "Fuzzy Sets," Information and Control, vol. 8, pp. 338-352, 1965.

Index Terms:
Pattern recognition, neural networks, rule systems, complexity, training, learning, a priori knowledge, fuzzy logic, Aristotelian logic
Leonid I. Perlovsky, "Conundrum of Combinatorial Complexity," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 6, pp. 666-670, June 1998, doi:10.1109/34.683784
Usage of this product signifies your acceptance of the Terms of Use.