This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters
September 1996 (vol. 18 no. 9)
pp. 912-926

Abstract—We present a new approach to structure learning in the field of Bayesian networks: We tackle the problem of the search for the best Bayesian network structure, given a database of cases, using the genetic algorithm philosophy for searching among alternative structures. We start by assuming an ordering between the nodes of the network structures. This assumption is necessary to guarantee that the networks that are created by the genetic algorithms are legal Bayesian network structures. Next, we release the ordering assumption by using a "repair operator" which converts illegal structures into legal ones. We present empirical results and analyze them statistically. The best results are obtained with an elitist genetic algorithm that contains a local optimizer.

[1] J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Mateo, Calif.: Morgan Kaufman, 1988.
[2] R. Neapolitan, Probabilistic Reasoning in Expert Systems: Theory and Algorithms, John Wiley&Sons, New York, 1990.
[3] F.V. Jensen, "Introduction to Bayesian Networks," Technical Report IR 93-2003, Dept. of Mathematics and Computer Science, Univ. of Aalborg, Denmark, 1993.
[4] C.K. Chow and C.N. Liu,"Approximating discrete probability distributions with dependence trees," IEEE Trans. Information Theory, vol. 14, no. 3, pp. 462-467, May 1968.
[5] G. Rebane and J. Pearl, "The Recovery of Causal Polytrees from Statistical Data," Uncertainty in Artificial Intelligence, vol. 3, pp. 175-182, 1989.
[6] S. Acid, L.M. De Campos, A. Gonzalez, R. Molina, and N. Perez de la Blanca, "Learning with CASTLE," Symbolic and Quantitative Approaches to Uncertainty, R. Kruse and P. Siegel, eds., Lecture Notes in Computer Science 548. Springer-Verlag, 1991.
[7] R.M. Fung and S.L. Crawford, "CONSTRUCTOR: A System for the Induction of Probabilistic Models," Proc. AAAI, pp. 762-769, 1990.
[8] E. Herskovits and G. Cooper, "Kutató: An Entropy-Driven System for Construction of Probabilistic Expert Systems from Databases," Report KSL-90-22, Knowledge Systems Laboratory, Medical Computer Science, Stanford Univ., 1990.
[9] G.F. Cooper and E. Herskovits, “A Bayesian Method for the Induction of Probabilistic Networks from Data,” Machine Learning, vol. 9, pp. 309–347, 1992.
[10] R.R. Bouckaert, "Probabilistic Network Construction Using the Minimum Description Length Principle," Lectures Notes in Computer Science 747, Symbolic and Quantitative Approaches to Reasoning and Uncertainty, ECSQARU '93, pp. 41-48, 1993.
[11] D. Wedelin, "Efficient Algorithms for Probabilistic Inference Combinatorial Optimization and the Discovery of Causal Structure from Data," doctoral dissertation, Chalmers Univ. of Tech nology, Göteborg, 1993.
[12] S.L. Lauritzen, B. Thiesson, and D.J. Spiegelhalter, "Diagnostic Systems Created by Model Selection Methods—A Case Study," Proc. Fourth Int'l Workshop Artificial Intelligence and Statistics, pp. 93-105, 1993.
[13] C.F. Aliferis and G.F. Cooper, "An Evaluation of an Algorithm for Inductive Learning of Bayesian Belief Networks Using Simulated Data Sets," Uncertainty in Artificial Intelligence, Proc. 10th Conf. , pp. 8-14, 1994.
[14] R.R. Bouckaert, "Optimizing Causal Orderings for Generating DAGs from Data," Uncertainty in Artificial Intelligence, Proc. Eighth Conf. , pp. 9-16, 1992.
[15] R.R. Bouckaert, "Properties of Bayesian Belief Networks Learning Algorithms," Proc. Uncertainty in Artificial Intelligence, 10th Ann. Conf. , pp. 102-109,Washington, D.C., 1994.
[16] D.M. Chickering, D. Geiger, and D. Heckerman, "Learning Bayesian Networks: Search Methods and Experimental Results," Fifth Int'l Workshop Artificial Intelligence and Statistics, pp. 112-128, 1995.
[17] D. Heckerman, D. Geiger, and D.M. Chickering, "Learning Bayesian Networks: The Combination of Knowledge and Statistical data," Technical Report MSR-TR-94-09, Microsoft, 1994.
[18] W. Lam and F. Bacchus, "Learning Bayesian Belief Networks. An Approach Based on the MDL Principle," Computational Intelligence, vol. 10, no. 4, 1994.
[19] W. Lam and F. Bacchus, "Using Causal Information and Local Measures to Learn Bayesian Networks," Uncertainty in Artificial Intelligence, Proc. Ninth Conf., pp. 243-250, 1993.
[20] D. Madigan, A.E. Raftery, J.C. York, J.M. Bradshaw, and R.G. Almond, "Strategies for Graphical Model Selection," Proc. Fourth Int'l Workshop Artificial Intelligence and Statistics, pp. 331-336, 1993.
[21] R. Mechling and M. Valtorta, "PaCCIN: A Parallel Constructor of Markov Networks," Proc. Fourth Int'l Workshop Artificial Intelligence and Statistics, pp. 405-410, 1993.
[22] G.M. Provan and M. Singh, "Learning Bayesian Networks Using Feature Selection," Preliminary Papers Fifth Int'l Workshop Artificial Intelligence and Statistics, pp. 450-456,Florida, 1995.
[23] G.M. Provan, "Model Selection for Diagnosis and Treatment Using Temporal Influence Diagrams," Proc. Fourth Int'l Workshop Artificial Intelligence and Statistics, pp. 469-480, 1995.
[24] M. Singh and M. Valtorta, "An Algorithm for the Construction of Bayesian Network Structures from Data," Uncertainty in Artificial Intelligence, Proc. Ninth Conf., pp. 259-265, 1993.
[25] J. Suzuki, "A Construction of Bayesian Networks from Databases Based on an MDL Principle," Uncertainty in Artificial Intelligence, Proc. Ninth Conf., pp. 266-273, 1993.
[26] P. Larrañaga and Y. Yurramendi, "Structure Learning Approaches in Causal Probabilistics Networks," Symbolic and Quantitative Approaches to Reasoning and Uncertainty, M. Clarke, R. Kruse, and S. Moral, eds., Lecture Notes in Computer Science 747, pp. 227-232. Springer-Verlag, 1993.
[27] H.J. Bremermann, M. Rogson, and S. Salaff, "Search by Evolution," Biophysics and Cybernetic Systems, M. Maxfield, A. Callahan, and L.J. Fogel, eds., pp. 157-167.Washington, D.C.: Spartan Books, 1965.
[28] I. Rechenberg, Optimierung technischer Systeme nach Prinzipien der biologischen Information.Stuttgart: Frommann Verlag, 1973 (in German).
[29] J.H. Holland, Adaptation in Natural and Artificial Systems. Univ. of Michigan Press, 1975.
[30] D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, Mass.: Addison-Wesley, 1989.
[31] Handbook of Genetic Algorithms, L. Davis, ed. New York: Van Nostrand Reinhold, 1991.
[32] L.J. Fogel, "Autonomous Automata," Ind. Res., vol. 4, pp. 14-19, 1962.
[33] H.-P. Schwefel, Numerische Optimierung von Computer-Modellen mittels der Evolutionsstrategie.Basel: Birkhäuser, 1977 (in German).
[34] A.E. Eiben, E.H.L. Aarts, and K.M. van Hee, "Global Convergence of Genetic Algorithms: An Infinite Markov Chain Analysis," Computing Science Notes, Eindhoven Univ. of Tech nology, 1990.
[35] C. Janikov and Z. Michalewicz, "On the Convergence Problem in Genetic Algorithms," UNCC technical report, 1990.
[36] R.F. Hartl, A Global Convergence Proof for a Class of Genetic Algorithms. Univ. of Tech nology, Vienna, 1990.
[37] W. Pedrycz, Fuzzy Neural Networks with Reference Neurons as Pattern Classifiers IEEE Trans. Neural Networks, vol. 3, no. 5, pp. 770-775, Sept. 1992.
[38] G. Rudolph, “Convergence Analysis of Canonical Genetic Algorithms,” IEEE Trans. Neural Networks, vol. 5, no. 1, pp. 96-101, 1994.
[39] P. Larrañaga and M. Poza, "Structure Learning of Bayesian Networks by Genetics Algorithms," Studies in Classification, Data Analysis, and Knowledge Organization, E. Diday, ed., pp. 300-306. Springer-Verlag, 1994.
[40] P. Larrañaga, R.H. Murga, M. Poza, and C.M.H. Kuijpers, "Structure Learning of Bayesian Networks by Hybrid Genetic Algorithms," Preliminary Papers Fifth Int'l Workshop Artificial Intelligence and Statistics, pp. 310-316, 1995.
[41] R.W. Robinson, "Counting Unlabeled Acyclic Digraphs," Lectures Notes in Mathematics 622: Combinatorial Mathematics V, C.H.C. Little, ed., pp. 28-43.New York: Springer-Verlag, 1977.
[42] K. Höffgen, "Learning and Robust Learning of Product Distribuctions," Technical Report 464, Fachbereich Informatik, Universität Dortmund, 1993.
[43] S.L. Lauritzen and D.J. Spiegelhalter, "Local Computations with Probabilities on Graphical Structures and Their Application on Expert Systems," J. Royal Statistical Soc. B, vol. 50, no. 2, pp. 157-224, 1988.
[44] M. Henrion, "Propagating Uncertainty in Bayesian Networks by Probabilistic Logic Sampling," Uncertainty in Artificial Intelligence, vol. 2, pp. 149-163, 1988.
[45] I.A. Beinlinch, H.J. Suermondt, R.M. Chavez, and G.F. Cooper, "The ALARM Monitoring System: A Case Study with Two Probabilistic Inference Techniques for Belief Networks," Proc. Second European Conf. Artificial Intelligence in Medicine, pp. 247-256, 1989.
[46] E.H. Herskovits, "Computer Based Probabilistic-Network Construction," doctoral dissertation, Medical Information Sciences, Stanford Univ., 1991.
[47] SPSS-X User's Guide. third edition, 1988.
[48] I. Matzkevich and B. Abramson, "The Topological Fusion of Bayes Nets," Proc. Eighth Conf. Uncertainty in Artificial Intelligence, pp. 191-198, 1992.
[49] U. Kjærulff, "A Computational Scheme for Reasoning in Dynamic Probabilistic Networks," Proc. Eighth Conf. Uncertainty in Artificial Intelligence, pp. 121-129, 1992.

Index Terms:
Bayesian network, genetic algorithm, structure learning, combinatorial optimization, performance analysis.
Citation:
Pedro Larrañaga, Mikel Poza, Yosu Yurramendi, Roberto H. Murga, Cindy M.H. Kuijpers, "Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 9, pp. 912-926, Sept. 1996, doi:10.1109/34.537345
Usage of this product signifies your acceptance of the Terms of Use.