This Article 
 Bibliographic References 
 Add to: 
On Compatible Priors for Bayesian Networks
September 1996 (vol. 18 no. 9)
pp. 901-911

Abstract—Given a Bayesian network of discrete random variables with a hyper-Dirichlet prior, a method is proposed for assigning Dirichlet priors to the conditional probabilities of structurally different networks. It defines a distance measure between priors which is to be minimized for the assignment process. Intuitively one would expect that if two models' priors are to qualify as being 'close' in some sense, then their posteriors should also be nearby after an observation. However one does not know in advance what will be observed next. Thus we are led to propose an expectation of Kullback-Leibler distances over all possible next observations to define a measure of distance between priors. In conjunction with the additional assumptions of global and local independence of the parameters [15], a number of theorems emerge which are usually taken as reasonable assumptions in the Bayesian network literature. The method is compared to the 'expansion and contraction' algorithm of [14], and is also contrasted with the results obtained in [7] who employ the additional assumption of likelihood equivalence which is not made here. A simple example illustrates the technique.

[1] S.A. Andersson, D. Madigan, and M.D. Perlman, "A Characterisation of Markov Equivalence Classes for Acyclic Digraphs," Tech. Report 287, Department of Statistics, Univ. Washington, Seattle, 1995.
[2] D. Chickering, "Search Operators for Learning Equivalence Classes of Bayesian Network Structures," Tech. Report R231, Cognitive Science Laboratory, Univ. California, Los Angeles, 1995.
[3] G.F. Cooper and E. Herskovits, “A Bayesian Method for the Induction of Probabilistic Networks from Data,” Machine Learning, vol. 9, pp. 309–347, 1992.
[4] R.G. Cowell, A.P. Dawid, and D.J. Spiegelhalter, "Sequential Model Criticism in Probabilistic Expert Systems," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, pp. 209-219, 1993.
[5] M. Frydenberg, "The Chain Graph Markov Property," Scandinavian J. Statistics, vol. 17, pp. 323-353, 1989.
[6] D. Heckerman and D. Geiger, "Likelihoods and Priors for Bayesian Networks," Technical Report MSR-TR-95-04, Microsoft Research, 1995.
[7] D. Heckerman, D. Geiger, and D.M. Chickering, "Learning Bayesian Networks: The Combination of Knowledge and Statistical Data," Technical Report MSR-TR-95-54, Microsoft Research, 1994.
[8] S.L. Lauritzen, "Mixed Graphical Association Models (with discussion)," Scandinavian J. Statistics, vol. 16, pp. 273-306, 1989.
[9] S.L. Lauritzen and D.J. Spiegelhalter, "Local Computations with Probabilities on Graphical Structures and Their Application to Expert Systems (with discussion)," J. Royal Statistical Soc., Series B, vol. 50, pp. 157-224, 1988.
[10] D. Nilsson, "An Algorithm for Finding the Most Probable Configurations of Discrete Variables that Are Specified in Probabilistic Expert Systems," MSc thesis, Department of Math. Statistics, Univ. Copenhagen, 1994.
[11] J. Pearl, Probabilistic Inference in Intelligent Systems.San Mateo, Calif.: Morgan Kaufmann, 1988.
[12] T. Verma and J. Pearl, "Equivalence and Synthesis of Causal Models," Proc. Sixth Conf. Uncertainty in Artificial Intelligence,Boston, Mass., pp. 220-227, 1990.
[13] D.J. Spiegelhalter and R.G. Cowell, "Learning in Probabilistic Expert Systems," Bayesian Statistics 4, eds. J.M. Bernardo, J.O. Berger, A.P. Dawid, and A.F.M. Smith, pp. 447-465.Oxford, England: Clarendon Press, 1992.
[14] D.J. Spiegelhalter, A.P. Dawid, S.L. Lauritzen, and R.G. Cowell, "Bayesian Analysis in Expert Systems," Statistical Science, vol. 8, pp. 219-247, 1993.
[15] D.J. Spiegelhalter and S.L. Lauritzen, "Sequential Updating of Conditional Probabilities on Directed Graphical Structures," Networks, vol. 20, pp. 579-605, 1990.
[16] J. Whittaker, Graphical Models in Applied Multivariate Analysis.New York: John Wiley and Sons, 1990.

Index Terms:
Bayesian networks, Dirichlet priors, Kullback-Leibler distance, local independence, global independence.
Robert G. Cowell, "On Compatible Priors for Bayesian Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 9, pp. 901-911, Sept. 1996, doi:10.1109/34.537344
Usage of this product signifies your acceptance of the Terms of Use.