This Article 
 Bibliographic References 
 Add to: 
Bayesian Network Refinement Via Machine Learning Approach
March 1998 (vol. 20 no. 3)
pp. 240-251

Abstract—A new approach to refining Bayesian network structures from new data is developed. Most previous work has only considered the refinement of the network's conditional probability parameters and has not addressed the issue of refining the network's structure. We tackle this problem by a machine learning approach based on a formalism known as the Minimum Description Length (MDL) principle. The MDL principle is well suited to this task since it can perform tradeoffs between the accuracy, simplicity, and closeness to the existent structure. Another salient feature of this refinement approach is the capability of refining a network structure using partially specified data. Moreover, a localization scheme is developed for efficient computation of the description lengths since direct evaluation involves exponential time resources.

[1] J.M. Agosta, "The Structure of Bayes Networks for Visual Recognition," R. Shacter, T.S. Levitt, L.N. Kanal, and J.F. Lemmer, eds., Uncertainty in Artificial Intelligence 4, pp. 397-405.New York: North-Holland, 1990.
[2] I.A. Beinlich, H.J. Suermondt, R.M. Chavez, and G.F. Cooper, "The ALARM Monitoring System: A Case Study With Two Probabilistic Inference Techniques for Belief Networks," Proc. Second European Conf. Artificial Intelligence in Medicine, pp. 247-256, 1989.
[3] C. Berzuini, R. Bellazzi, and D. Spiegelhalter, "Bayesian Networks Applied to Therapy Monitoring," Proc. Conf. Uncertainty in Artificial Intelligence, pp. 35-43, 1991.
[4] W. Buntine, "Theory Refinement on Baysian Networks," Proc. Conf. Uncertainty in Artificial Intelligence, pp. 52-60, 1991.
[5] R.M. Chavez and G.F. Cooper, "A Randomized Approximation Algorithm for Probabilistic Inference on Bayesian Belief Networks," Networks, vol. 20, pp. 661-685, 1990.
[6] G.F. Cooper, "The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks," Artificial Intelligence, vol. 42, pp. 393-405, 1990.
[7] G.F. Cooper and E. Herskovits, "A Bayesian Method for the Induction of Probabilistic Networks From Data," Machine Learning, vol. 9, pp. 309-347, 1992.
[8] R.G. Cowell, A.P. Dawid, and D.J. Spiegelhalter, "Sequential Model Criticism in Probabilistic Expert Systems," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, pp. 209-219, 1993.
[9] P. Dagum and M. Luby, "Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard," Artificial Intelligence, vol. 60, pp. 141-153, 1993.
[10] F.J. Diez, "Parameter Adjustment in Bayes Networks, the Generalized Noisy OR-Gate," Proc. Conf. Uncertainty in Artificial Intelligence, pp. 99-105, 1993.
[11] D. Heckerman, J. Breese, and K. Rommelse, "Sequential Troubleshooting Under Uncertainty," Proc. Fifth Int'l Workshop Principles of Diagnosis, pp. 121-130, 1994.
[12] D. Heckerman, D. Geiger, and D. Chickering, "Learning Bayesian Networks: The Combination of Knowledge and Statistical Data," Machine Learning, vol. 20, pp. 197-243, 1995.
[13] M. Henrion, "Towards Efficient Inference in Multiply Connected Belief Networks," Influence Diagrams, Belief Nets, and Decision Analysis, pp. 385-407.Chichester, England: Wiley, 1990.
[14] W. Lam and F. Bacchus, "Using Causal Information and Local Measure to Learn Bayesian Networks," Proc. Conf. Uncertainty in Artificial Intelligence, pp. 243-250, 1993.
[15] W. Lam and F. Bacchus, "Learning Bayesian Belief Networks—An Approach Based on the MDL Principle," Computational Intelligence, vol. 10, no. 3, pp. 269-293, 1994.
[16] S.L. Lauritzen and D.J. Spiegelhalter, "Local Computation With Probabilities on Graphical Structures, and Their Applications to Expert Systems," J. Royal Statistical Soc., Series B, vol. 50, pp. 157-244, 1988.
[17] A. Milosavljevic and J. Jurka, "Discovery by Minimal Length Encoding: A Case Study in Molecular Evolution," Machine Learning, vol. 12, pp. 69-87, 1993.
[18] R. Musick, "Minimal Assumption Distribution Propagation in Belief Networks," Proc. Conf. Uncertainty in Artificial Intelligence, pp. 251-258, 1993.
[19] S.A. Musman, L.W. Chang, and L.B. Booker, "Application of a Real-Time Control Strategy for Bayesian Belief Networks to Ship Classification Problem Solving," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 7, no. 3, pp. 513-526, 1993.
[20] J. Pearl, "Evidential Reasoning Using Stochastic Simulation of Causal Models," Artificial Intelligence, vol. 32, pp. 245-257, 1987.
[21] J. Pearl and T.S. Verma, "A Theory of Inferred Causation," Proc. Second Int'l Conf. Principles of Knowledge Representation and Reasoning, pp. 441-452, 1991.
[22] J.R. Quinlan and R.L. Rivest, "Inferring Decision Trees Using the Minimum Description Length Principle," Information and Computation, vol. 80, pp. 227-248, 1989.
[23] J. Rissanen, "Modeling by Shortest Data Description," Automatica, vol. 14, pp. 465-471, 1978.
[24] J. Rissanen, Stochastic Complexity in Statistical Inquiry. World Scientific, 1989.
[25] R.W. Robinson, "Counting Unlabeled Acyclic Digraphs," Proc. Fifth Australian Conf. Combinatorial Math., pp. 28-43, 1976.
[26] S. Russell, J. Binder, D. Koller, and K. Kanazawa, "Local Learning in Probabilistic Networks With Hidden Variables," Proc. Int'l Joint Conf. Artifical Intelligence (IJCAI), pp. 1,146-1,152, 1995.
[27] R.D. Shachter and M.A. Peot, "Simulation Approaches to General Probabilistic Inference on Belief Networks," M. Henrion, R. Shacter, L. N. Kanal, and J. F. Lemmer, eds., Uncertainty in Artificial Intelligence 5, pp. 221-231.New York: North-Holland, 1990.
[28] D.J. Spiegelhalter and R.G. Cowell, "Learning in Probabilistic Expert Systems," Bayesian Statistics 4, pp. 447-465. Oxford Univ. Press, 1992.
[29] D.J. Spiegelhalter and S.L. Lauritzen, "Sequential Updating of Conditional Probabilities on Directed Graphical Structures," Networks, vol. 20, pp. 579-605, 1990.
[30] P. Spirtes and C. Meek, "Learning Bayesian Networks With Discrete Variables From Data," Proc. First Int'l Conf. Knowledge Discovery and Data Mining, pp. 294-299, 1995.
[31] T. Verma and J. Pearl, "Equivalence and Synthesis of Causal Models," Proc. Conf. Uncertainty in Artificial Intelligence, pp. 220-227, 1990.

Index Terms:
Knowledge base refinement, uncertainty reasoning, Bayesian networks, machine learning, data mining.
Wai Lam, "Bayesian Network Refinement Via Machine Learning Approach," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 240-251, March 1998, doi:10.1109/34.667882
Usage of this product signifies your acceptance of the Terms of Use.