The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2008 vol.20)
pp: 628-640
ABSTRACT
Structure learning of Bayesian networks is a well-researched but computationally hard task. We present an algorithm that integrates an information theory-based approach and a scoring function-based approach for learning structures of Bayesian networks. Our algorithm also makes use of basic Bayesian network concepts like d-separation and Markov independence. We show that the proposed algorithm is capable of handling networks with a large number of variables. We present the applicability of the proposed algorithm on four standard network datasets and also compare its performance and computational efficiency with other standard structure learning methods. The experimental results show that our method can efficiently and accurately identify complex network structures from data.
INDEX TERMS
classification, data mining, Machine learning
CITATION
Xue-Wen Chen, Gopalakrishna Anantha, Xiaotong Lin, "Improving Bayesian Network Structure Learning with Mutual Information-Based Node Ordering in the K2 Algorithm", IEEE Transactions on Knowledge & Data Engineering, vol.20, no. 5, pp. 628-640, May 2008, doi:10.1109/TKDE.2007.190732
REFERENCES
[1] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
[2] O. Pietquin and T. Dutoit, “A Probabilistic Framework for Dialog Simulation and Optimal Strategy Learning,” IEEE Trans. Speech and Audio Processing, vol. 14, no. 2, pp. 589-599, 2005.
[3] J. Agosta, “The Structure of Bayes Networks for Visual Recognition,” Uncertainty in Artificial Intelligence, T.L.R. Shacter, L.N.Kanal, and J.F. Lemmer, eds., vol. 4, pp. 397-405, 1990.
[4] D. Heckerman, “Probabilistic Similarity Networks,” Networks, vol. 20, pp. 607-636, 1990.
[5] D. Nikovski, “Constructing Bayesian Networks for Medical Diagnosis from Incomplete and Partially Correct Statistics,” IEEE Trans. Knowledge and Data Eng., vol. 12, no. 4, pp. 509-516, July/Aug. 2000.
[6] N. Friedman, M. Linial, I. Nachman, and D. Pe'er, “Using Bayesian Networks to Analyze Expression Data,” J. Computational Biology, vol. 7, pp. 601-620, 2000.
[7] J. Yu, V. Smith, P. Wang, A. Hartemink, and E. Jarvis, “Advances to Bayesian Network Inference for Generating Causal Networks from Observational Biological Data,” Bioinformatics, vol. 20, pp.3594-3603, 2004.
[8] L. de Campos, “Independency Relationships and Learning Algorithms for Singly Connected Networks,” J. Experimental and Theoretical Artificial Intelligence, vol. 10, pp. 511-549, 1998.
[9] L. de Campos and J. Huete, “A New Approach for Learning Belief Networks Using Independence Criteria,” Int'l J. Approximate Reasoning, vol. 24, pp. 11-37, 2000.
[10] J. Pearl and T. Verma, “Equivalence and Synthesis of Causal Models,” Proc. Sixth Conf. Uncertainty in Artificial Intelligence, 1990.
[11] W. Lam and F. Bacchus, “Learning Bayesian Belief Networks. An Approach Based on the MDL Principle,” Computational Intelligence, vol. 10, pp. 269-293, 1994.
[12] G. Cooper and E. Herskovits, “A Bayesian Method for the Induction of Probabilistic Networks from Data,” Machine Learning, vol. 9, pp. 309-347, 1992.
[13] D. Heckerman, D. Geiger, and D. Chickering, “Learning Bayesian Networks: The Combination of Knowledge and Statistical Data,” Machine Learning, vol. 20, pp. 197-243, 1995.
[14] P. Spirtes, C. Glymour, and R. Scheines, “Causation, Prediction and Search,” Lecture Notes in Statistics 81, Springer, 1993.
[15] N. Wermuth and S. Lauritzen, “Graphical and Recursive Models for Contingence Tables,” Biometrika, vol. 72, pp. 537-552, 1983.
[16] L. Campos and J. Huete, “On the Use of Independence Relationships for Learning Simplified Brief Networks,” Int'l J. Intelligent Systems, vol. 12, pp. 495-522, 1997.
[17] D. Geiger, A. Paz, and J. Pearl, “Learning Causal Trees from Dependence Information,” Proc. Eighth Nat'l Conf. Artificial Intelligence (AAAI), 1990.
[18] D. Geiger, A. Paz, and J. Pearl, “Learning Simple Causal Structures,” Int'l J. Intelligent Systems, vol. 8, pp. 231-247, 1993.
[19] J. Cheng, D. Bell, and W. Liu, “An Algorithm for Bayesian Belief Network Construction from Data,” Proc. Sixth Int'l Workshop Artificial Intelligence and Statistics, 1997.
[20] C. Meek, “Causal Inference and Causal Explanation with Background Knowledge,” Proc. 11th Conf. Uncertainty in Artificial Intelligence, 1995.
[21] W. Buntine, “Theory Refinement on Bayesian Networks,” Proc. Seventh Conf. Uncertainty in Artificial Intelligence, pp. 52-60, 1991.
[22] D. Chickering, “Learning Equivalence Classes on Bayesian-Network Structures,” J. Machine Learning Research, vol. 2, pp.445-498, 2002.
[23] D. Chickering, “Optimal Structure Identification with Greedy Search,” J. Machine Learning Research, vol. 3, pp. 507-554, 2002.
[24] W. Buntine, “A Guide to the Literature on Learning Probabilistic Networks from Data,” IEEE Trans. Knowledge and Data Eng., vol. 8, pp. 195-210, 1996.
[25] N. Friedman and D. Koller, “Being Bayesian about Network Structure: A Bayesian Approach to Structure Discovery in Bayesian Networks,” Machine Learning, vol. 50, pp. 95-125, 2003.
[26] D. Heckerman, “A Tutorial on Learning Bayesian Networks,” Technical Report MSR-TR-95-06, Microsoft Research, 1996.
[27] J. Suzuki, “A Construction of Bayesian Networks from Databases Based on an MDL Principle,” Proc. Ninth Conf. Uncertainty in Artificial Intelligence, pp. 266-273, 1993.
[28] R. Bouckaert, “Belief Networks Construction Using the Minimum Description Length Principle,” Lecture Notes in Computer Science 747, pp. 41-48, 1993.
[29] N. Friedman and M. Goldszmidt, “Learning Bayesian Networks with Local Structure,” Proc. 12th Conf. Uncertainty in Artificial Intelligence, 1996.
[30] C. Chow and C. Liu, “Approximating Discrete Probability Distributions with Dependence Trees,” IEEE Trans. Information Theory, vol. 14, pp. 462-467, 1968.
[31] M. Koivisto and K. Sood, “Exact Bayesian Structure Discovery in Bayesian Networks,” J. Machine Learning Research, vol. 5, pp. 549-573, 2004.
[32] D. Chickering, “Learning Bayesian Networks Is NP-Complete,” Learning from Data: Artificial Intelligence and Statistics V, D. Fisher and H. Lenz, eds., pp. 121-130, Springer, 1996.
[33] D. Chickering, D. Heckerman, and C. Meek, “Large-Sample Learning of Bayesian Networks Is NP-Hard,” J. Machine Learning Research, vol. 5, pp. 1287-1330, 2004.
[34] S. Acid and L. de Campos, “Searching for Bayesian Network Structures in the Space of Restricted Acyclic Partially Directed Graphs,” J. Artificial Intelligence Research, vol. 18, pp. 445-490, 2003.
[35] R. Castelo and T. Kocka, “On Inclusion-Driven Learning of Bayesian Networks,” J. Machine Learning Research, vol. 4, pp. 527-574, 2003.
[36] P. Larranaga, C. Kuijpers, R. Murga, and Y. Yurramendi, “Learning Bayesian Network Structures by Searching for the Best Ordering with Genetic Algorithms,” IEEE Trans. Systems, Man, and Cybernetics, vol. 26, pp. 487-493, 1996.
[37] P. Larranaga, M. Poza, Y. Yurramendi, R. Murga, and C. Kuijpers, “Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, pp. 912-926, 1996.
[38] D. Chickering, D. Geiger, and D. Heckerman, “Learning Bayesian Networks: Search Methods and Experimental Results,” Preliminary Papers Fifth Int'l Workshop Artificial Intelligence and Statistics, 1995.
[39] D. Madigan and J. York, “Bayesian Graphical Models for Discrete Data,” Int'l Statistical Rev., vol. 63, pp. 215-232, 1995.
[40] F. Jensen, An Introduction to Bayesian Networks. UCL Press, 1996.
[41] J. Proakis, Digital Communications. McGraw-Hill, 2000.
[42] G. Chartrand and L. Lesniak, Graphs and Digraph. Chapman and Hall/CRC, 2005.
[43] S. Lauritzen and D. Spiegelhalter, “Local Computations with Probabilities on Graphical Structures and Their Application on Expert Systems,” J. Royal Statistical Soc., vol. 50, pp. 157-224, 1988.
[44] I. Beinlich, G. Suermondt, R. Chavez, and G. Cooper, “The ALARM Monitoring System: A Case Study with Two Probabilistic Inference Techniques for Belief Networks,” Proc. Second European Conf. Artificial Intelligence in Medicine, 1989.
[45] B. Abramson, J. Brown, W. Edwards, A. Murphy, and R. Winkler, “Hailfinder: A Bayesian System for Forecasting Severe Weather,” Int'l J. Forecasting, vol. 12, pp. 57-71, 1996.
[46] J. Cheng, R. Grenier, J. Kelly, D. Bell, and W. Liu, “Learning Bayesian Networks from Data: An Information-Theory Based Approach,” Artificial Intelligence, vol. 137, pp. 43-90, 2002.
[47] http:/bnt.sourceforge.net/, 2006.
[48] P. Murphy and D. Aha UCI Repository of Machine Learning Database, http://www.ics.uci.edu/~mlearnMLRepository.html , 1995.
18 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool