This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Population-Based Learning: A Method for Learning from Examples Under Resource Constraints
October 1992 (vol. 4 no. 5)
pp. 454-474

A learning model for designing heuristics automatically under resource constraints is studied. The focus is on improving performance-related heuristic methods (HMs) in knowledge-lean application domains. It is assumed that learning is episodic, that the performance measures of an episode are dependent only on the final state reached in evaluating the corresponding test case, and that the aggregate performance measures of the HMs involved are independent of the order of evaluation of test cases. The learning model is based on testing a population of competing HMs for an application problem, and switches from one to another dynamically, depending on the outcome of previous tests. Its goal is to find a good HM within the resource constraints, with proper tradeoff between cost and quality. It extends existing work on classifier systems by addressing issues related to delays in feedback, scheduling of tests of HMs under limited resources, anomalies in performance evaluation, and scalability of HMs. Experience in applying the learning method is described.

[1] A. Barr and E. A. Feigenbaum,The Handbook of Artificial Intelligence, Vols. 1-3. Los Altos, CA: William Kaufmann, 1982.
[2] R. E. Bechhofer, C. W. Dunnett, and M. Sobel, "A two-sample multiple decision procedure for ranking means of normal populations with a common unknown variance,"Biometrika, vol. 41, pp. 170-176, 1954.
[3] R. E. Bechhofer, "A single-sample multiple decision procedure for ranking means of normal populations with known variances,"Ann. Math. Statist., vol. 25, no. 1, pp. 16-39, Mar. 1954.
[4] R. E. Bechhofer, J. Kiefer, and M. Sobel,Sequential Identification and Ranking Procedures. Chicago, IL: University of Chicago, 1968.
[5] R. E. Bechhofer, A. J. Hayter, and A. C. Tamhane, "Designing experiments for selecting the largest normal mean when the variances are known and unequal: Optimal sample size allocation,"J. Statist. Planning Inference, vol. 28, pp. 271-289, 1991.
[6] L.B. Booker, D.E. Goldberg, and J.H. Holland, "Classifier Systems and Genetic Algorithms,"Artificial Intelligence, Vol. 40, Sept. 1989, pp. 235-282.
[7] T. G. Dietterich and B. G. Buchanan, "The role of critic in learning systems," Tech. Rep. STAN-CS-81-891, Stanford Univ., CA, Dec. 1981.
[8] S. E. Fahlman, "Faster-learning variations on back-propagation: An empirical study," inProc. Connectionist Models Summer School, D. Touretzky, G. E. Hinton, and T. J. Sejnowski, eds., Palo Alto, CA, 1988, pp. 38-51.
[9] S. E. Fahlman and C. Lebiere, "The cascade-correlation learning architecture,"Advances in Neural Information Processing Systems, 2 ed., D. S. Touretzky, ed. Palo Alto, CA: Morgan Kaufmann, 1990, pp. 524-532.
[10] J. M. Fitzpatrick and J. J. Grefenstette, "Genetic algorithms in noisy environments," inMach. Learning, vol. 3, no. 2/3, pp. 101-120, Oct. 1988.
[11] M. P. Georgeff, "Strategies in heuristic search."Artif. Intell., vol. 20, pp. 393-425, 1983.
[12] R. Gnanadesikan,Methods for Statistical Data Analysis of Multivariate Observations. New York: Wiley, 1977.
[13] J. J. Grefenstette, "Credit assignment in rule discovery systems based on genetic algorithms,"Mach. Learning, vol. 3, no. 2/3, pp. 225-246, Oct. 1988.
[14] J. J. Grefenstette, C. L. Ramsey, and A. C. Schultz, "Learning sequential decision rules using simulation models and competition,"Mach. Learning, vol. 5, pp. 355-381, 1990.
[15] S. S. Gupta and S. Panchapakesan, "Sequential ranking and selection procedures," inHandbook of Sequential Analysis, B. K. Ghosh and P. K. Sen, eds. New York: Marcel Dekker, 1991, pp. 363-380.
[16] J. H. Holland,Adaptation in Natural and Artificial Systems. Ann Arbor, MI: University of Michigan Press, 1975.
[17] J. H. Holland, "Properties of the bucket brigade algorithm," inProc. Int. Conf. Genetic Algorithms and Their Applications. 1985, pp. 1-7.
[18] A. Ieumwananonthachai, A. N. Aizawa, S. R. Schwartz, B. W. Wah, and J. C. Yan, "Intelligent mapping of communicating processes in distributed computing systems," inProc. Supercomputing 91, Albuquerque, NM, Nov. 1991, pp. 512-521.
[19] A. Ieumwananonthachai, "Intelligent process mapping through systematic improvement of heuristics,"J. Parallel Distributed Comput., vol. 15, pp. 118-142, June 1992.
[20] A. Ieumwananonthachai and B. W. Wah, "Parallel statistical selection in multiprocessors," inProc. Int. Conf. on Parallel Processing, 1992. pp. 190-194.
[21] R.A. Johnson and D.W. Wichern,Applied Multivariate Statistical Analysis, Prentice Hall, Englewood Cliffs, N.J., 1992.
[22] A. H. Klopf, "Drive-reinforcement learning: A real-time learning mechanism for unsupervised learning." inProc. Int. Conf. on Neural Networks, vol. II, 1987, pp. 441-445.
[23] M. M. Kokar and W. W. Zadrozny, "A logical model of machine learning," inProc. First Workshop on Change of Representation, 1988.
[24] J. K. Kruschke, "Creating local and distributed bottlenecks in hidden layers of back-propagation networks." inProc. Connectionist Models Summer School, 1988, pp. 120-126.
[25] P. Langley, "Learning to search: From weak methods to domain-specific heuristics."Cognitive Sci., vol. 9, pp. 217-260, 1985.
[26] D. B. Lenat, "Theory formation by heuristic search; The nature of heuristics II: Background and examples."Artif. Intell., vol. 21, pp. 31-59, 1983.
[27] D. Marr,Vision. New York: Freeman, 1982.
[28] J. McClelland and D. Rumelhart,Explorations in Parallel Distributed Processing. Cambridge, MA: MIT Press, 1988.
[29] P. Mehra and B. W. Wah, "Physical-level synthetic workload generation for load-balancing experiments," inProc. First Symp. on High Performance Distributed Computing, Syracuse, NY, Oct. 1992.
[30] P. Mehra and B. W. Wah, "Adaptive load-balancing strategies for distributed systems," inProc. 2nd Int. Conf. on Systems Integration, June 1992, pp.666-675.
[31] R. S. Michalski, "A theory and methodology of inductive learning," inMachine Learning, R. S. Michalski, J. G. Carbonell, and T. M. M. Tioga, eds. Los Altos, CA: Morgan Kaufman, 1983.
[32] R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Eds.,Machine Learning: An Artificial Intelligence Approach, vol. 2. Los Altos, CA: Morgan Kaufmann, 1986.
[33] M. Minsky, "Steps toward artificial intelligence,"Computers and Thought, E. A. Feigenbaum and J. Feldman, eds. New York: McGraw-Hill, 1963, pp. 406-450.
[34] T. M. Mitchell, P. E. Utgoff, B. Nudel, and R. Benerji, "Learning problem-solving heuristics through practice, " inProc. 7th Int. Joint Conf. on Artificial Intelligence, 1981, pp. 127-134.
[35] T. M. Mitchell, "Learning and problem solving," inProc. 8th Int. Joint Conf. on Artificial Intelligence, Aug. 1983, pp. 1139-1151.
[36] T. M. Mitchell, P. E. Utgoff, and R. B. Banerji, "Learning by experimentation: Acquiring and refining problem-solving heuristics," in R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, eds.Machine Learning. Palo Alto, CA: Tioga, 1983.
[37] M. C. Mozer and P. Smolensky, "Using relevance to reduce network size automatically,"Connection Sci., vol. 1, no. 1, pp. 3-16, 1989.
[38] A. Newell, J. C. Shaw, and H. A. Simon, "Programming the logic theory machine," inProf. 1957 Western Joint Computer Conf., 1957, pp. 230-240.
[39] D. Nguyen and B. Widrow, "The truck backer-upper: An example of self-learning in neural networks," inProc. Int. Joint Conf. on Neural Networks, vol. II, 1989, pp. 357-363.
[40] J. Pearl, "On the discovery and generation of certain heuristics,"AI Mag., pp. 23-33, Winter/Spring 1983.
[41] J. Pearl,Heuristics: Intelligent Search Strategies for Computer Problem Solving. Reading, Mass: Addison-Wesley, 1984.
[42] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representation by error propagation,"Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vols. 1 and 2. Cambridge, MA: MIT Press, 1986.
[43] S. Russell and E. Wefald, "Principles of metareasoning,"Artif. Intell., vol. 49, pp. 361-395, 1991.
[44] A. L. Samuel, "Some studies in machine learning using the game of checkers,"IBM J. Res. Develop., vol. 3, pp. 210-229, 1959.
[45] A. L. Samuel, "Some studies in machine learing using the game of checkers II--Recent progress,"IBM J. Res. Develop., vol. 11, no. 6, pp. 601-617, 1967.
[46] S. R. Schwartz, "Resource constrained parameter tuning applied to stereo vision," MSc. thesis, Dep. Elect. Comput. Eng., Univ. of Illinois, Urbana, IL, Aug. 1991.
[47] D. H. Bailey, "Vector computer memory bank contention,"IEEE Trans. Computers, vol. C-36, pp. 293-298, Mar. 1987.
[48] D. Sleeman, P. Langley, and T. M. Mitchell, "Learning from solution paths: An approach to the credit assignment problem."AI Mag., vol. 3, 1982, pp. 48-52.
[49] G. Smith, "Back propagation with dynamic topology and simple activation functions," Tech. Rep. TR 90-12, School Inform. Sci. Technol., Flinders Univ. of South Australia, Adelaide, 1990.
[50] R. G. Smith, T. M. Mitchell, R. A. Chestek, and B. G. Buchanan, "A model for learning systems," inProc. 5th Int. Joint Conf. on Artificial Intelligence, Aug. 1977, pp. 338-343.
[51] S. F. Smith, "Flexible learning of problem solving heuristics through adaptive search." inProc. Int. Joint Conf. on Artificial Intelligence, 1983, pp. 422-425.
[52] C. Stein, "The selection of the largest of a number of means,"Ann. Math. Statist., vol. 19, pp. 429, 1948.
[53] R. S. Sutton, "Temporal credit assignment in reinforcement learning," Ph.D. dissertation, Univ. of Massachusetts, Amherst, MA, Feb. 1984.
[54] R. S. Sutton and A. G. Barto, "Toward a modern theory of adaptive networks: Expectation and prediction,"Psychological Rev., vol. 88, no. 2, pp. 135-170, 1984.
[55] B. G. Tabachnick and L. S. Fidell,Using Multivariate Statistics, second ed. New York: Harper and Row, 1989.
[56] G. Tesauro, "Connectionist learning of expert backgammon evaluations,"Mach. Learning, pp. 200-206, 1988.
[57] Y. L. Tong and D. E. Wetzell, "Allocation of observations for selecting the best normal population," inDesign of Experiments: Ranking and Selection, T. J. Santner and A. C. Tamhane, eds. New York: Marcel Dekker, 1984, pp. 213-224.
[58] B. W. Wah and H. Kriplani, "Resource constrained design of artificial neural networks," inProc. Int. Joint Conf. on Neural Networks, IEEE, San Diego, CA, June 1990, pp. 269-279.
[59] E. H. Wefald and S. J. Russell, "Adaptive learning of decision-theoretic search control knowledge,"Mach. Learning, pp. 408-411, 1989.
[60] B. Widrow, N. K. Gupta, and S. Maitra, "Punish/reward: Learning with a critic in adaptive threshold systems."IEEE Trans. Syst., Man, Cybern., vol. SMC-3, pp. 455-465, 1973.
[61] D. C. Wilkins, "Knowledge base refinement using apprenticeship learning techniques," inProc. Nat. Conf. on Artificial Intelligence AAAI-88, 1988.
[62] R. J. Williams, "On the use of backpropagation in associative reinforcement learning," inProc. Int. Conf. on Neural Networks, vol. I, July 1988, pp. 263-270.
[63] S. W. Wilson, "Hierarchical credit allocation in classifier systems," inGenetic Algorithms and Simulated Annealing, Research Notes in Artificial Intelligence, L. Davis, ed. London, U.K.: Pitman, 1987.
[64] J. C. Yan and S. F. Lundstrom, "The post-game analysis framework--Developing resource management strategies for concurrent systems."IEEE Trans. Knowl. Data Eng., vol. 1, pp. 293-309, Sept. 1989.
[65] C. F. Yu and B. W. Wah, "Learning dominance relations in combinatorial search problems,"IEEE Trans. Software Eng., vol. SE-14, pp. 1155-1175, Aug. 1988.
[66] S. Zhou, "Performance studies of dynamic load balancing in distributed systems," Tech. Rep. UCB/CSD 87/376, Comput. Sci. Div., Univ. of California, Berkeley, CA, 1987.

Index Terms:
population based learning; feedback delays; test scheduling; learning from examples; resource constraints; performance-related heuristic methods; knowledge-lean application; performance measures; cost; quality; classifier systems; limited resources; performance evaluation; scalability; learning by example
Citation:
B.W. Wah, "Population-Based Learning: A Method for Learning from Examples Under Resource Constraints," IEEE Transactions on Knowledge and Data Engineering, vol. 4, no. 5, pp. 454-474, Oct. 1992, doi:10.1109/69.166988
Usage of this product signifies your acceptance of the Terms of Use.