This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Framework for Learning in Search-Based Systems
July/August 1998 (vol. 10 no. 4)
pp. 563-575

Abstract—In this paper, we provide an overall framework for learning in search-based systems that are used to find optimum solutions to problems. This framework assumes that prior knowledge is available in the form of one or more heuristic functions (or features) of the problem domain. An appropriate clustering strategy is used to partition the state space into a number of classes based on the available features. The number of classes formed will depend on the resource constraints of the system. In the training phase, example problems are run using a standard admissible search algorithm. In this phase, heuristic information corresponding to each class is learned. This new information can be used in the problem-solving phase by appropriate search algorithms so that subsequent problem instances can be solved more efficiently. In this framework, we also show that heuristic information of forms other than the conventional single-valued underestimate value can be used, since we maintain the heuristic of each class explicitly. We show some novel search algorithms that can work with some such forms. Experimental results have been provided for some domains.

[1] J. Pearl and J.H. Kim, "Studies in Semi-Admissible Heuristics," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 4, no. 4, pp. 392-399, 1982.
[2] J. Pearl, Heuristics: Intelligent Search Strategies for Computer Problem Solving, Addison-Wesley, Reading, Mass., 1984.
[3] A.L. Samuel, "Some Studies in Machine Learning Using the Game of Checkers," in Computers and Thought, E. Feigenbaum and J. Feldman, eds., McGraw-Hill, New York, 1963, pp. 71-105.
[4] A.L. Samuel, "Some Studies in Machine Learning Using the Game of Checkers II—Recent Progress," IBM J. Research and Development, vol. 11, pp. 601-607, 1967.
[5] H. Berliner, "On the Construction of Evaluation Functions for Large Domains," Proc. Sixth Int'l Joint Conf. Artificial Intelligence, pp. 53-55, 1977.
[6] H. Berliner, "The B* Tree Search Algorithm: A Best-First Proof Procedure," Artificial Intelligence, vol. 12, pp. 23-40, 1979.
[7] J. Christensen and R.E. Korf, "A Unified Theory of Heuristic Evaluation Function Learning," AAAI, 1986.
[8] K. Lee and S. Mahajan, “A Pattern Classification Approach to Evaluation Function Learning,” Artificial Intelligence, vol. 36, no. 1, pp. 1-25, 1988.
[9] L.A. Rendell, "A New Basis for State-Space Learning Systems and a Successful Implementation," Artificial Intelligence, vol. 20, pp. 369-392, 1983.
[10] A. Bramanti-Gregor and H.W. Davis, "Learning Admissible Heuristics while Solving Problems," Proc. 12th Int'l Joint Conf. Artificial Intelligence, 1991.
[11] A. Bramanti-Gregor and H.W. Davis, "Strengthening Heuristics for Lower Cost Optimal and Near Optimal Solution in A* Search," Proc. European Conf. Artificial Intelligence, pp. 6-10, 1992.
[12] A. Bramanti-Gregor and H.W. Davis, "The Statistical Learning of Accurate Heuristics," Proc. 13th Int'l Joint Conf. Artificial Intelligence, pp. 1,079-1,085, 1993.
[13] R.S. Sutton, "Learning to Predict by the Methods of Temporal Differences," Machine Learning, vol. 3, pp. 9-44, 1988.
[14] C.J. Watkins and P. Dayan, "Q Learning," Machine Learning, vol. 8, pp. 279-292, 1992.
[15] G. Tesauro and T.J. Sejnowski, "A Parallel Network that Learns to Play Backgammon," Artificial Intelligence, vol. 39, pp. 357-390, 1989.
[16] G. Tesauro, "Practical Issues in Temporal Difference Learning," Machine Learning, special issue on reinforcement learning, vol. 8, 1992.
[17] B.W. Wah, A. Ieumwananonthachai, L.-C. Chu, and A.N. Aizawa, "Genetics-Based Learning of New Heuristics: Rational Scheduling of Experiments and Generalization," IEEE Trans. Knowledge and Data Eng., vol. 7, no. 5, pp. 763-785, 1995.
[18] P. Langley, J.H. Gennari, and W. Iba, "Hill-Climbing Theories of Learning," Proc. Fourth Int'l Workshop Machine Learning, pp. 312-323, Morgan Kaufmann, 1987.
[19] W. Iba, J. Wogulis, and P. Langley, "Trading Off Simplicity and Coverage in Incremental Concept Learning," Proc. Fifth Int'l Conf. Machine Learning, pp. 73-79, Morgan Kaufmann, 1988.
[20] O. Hansson and A. Mayer, "Probabilistic Heuristic Estimates," Annals Math. and Artificial Intelligence, vol. 2, pp. 209-220, 1990.
[21] G. Politowski, "On the Construction of Heuristic Functions," PhD thesis, Univ. of California, Santa Cruz, 1986.
[22] N.J. Nilsson, Principles of Artificial Intelligence. Morgan Kaufmann, 1980.
[23] S. Sarkar, "New Approaches to Heuristic Search Involving Learning and Approximations," PhD thesis, Indian Inst. of Tech nology, Kharagpur, India, 1996.
[24] S. Sarkar, S. Ghose, and P.P. Chakrabarti, "Continuous Learning of the Distribution of Heuristic Estimates while Solving Problems," Proc. Fourth Nat'l Seminar Theoretical Computer Science,Kanpur, India, 1994.
[25] S. Sarkar, P.P. Chakrabarti, and S. Ghose, "Learning while Solving Problems in Best First Search," IEEE Trans. Systems, Man, and Cybernetic, accepted for publication.

Index Terms:
Clustering, heuristic features, learning, search algorithms.
Citation:
Sudeshna Sarkar, P.p. Chakrabarti, Sujoy Ghose, "A Framework for Learning in Search-Based Systems," IEEE Transactions on Knowledge and Data Engineering, vol. 10, no. 4, pp. 563-575, July-Aug. 1998, doi:10.1109/69.706057
Usage of this product signifies your acceptance of the Terms of Use.