This Article 
 Bibliographic References 
 Add to: 
Constructing Efficient Belief Network Structures With Expert Provided Information
February 1996 (vol. 8 no. 1)
pp. 134-143

Abstract—We present a technique to construct efficient belief network structures for application areas where large amounts of data are available and information on the ordering of the variables can be obtained from domain experts. We identify classes of networks that are efficient for propagating beliefs. We formulate the problem as one of determining the belief network representation from a given class that best represents the data. We use the I-Divergence measure which is known to have certain desirable properties for evaluating different approximations. We present some theoretical findings that characterize the nature of solutions that are obtained. These theoretical results lead to an efficient solution procedure for finding the best network representation. We also discuss other information that may be reasonably obtained from experts, and show how such information leads to improving the efficiency of the technique to find the best network structure.

[1] G.W. Brier,"Verification of forecasts expressed in terms of probability," Monthly Weather Review, vol. 78, no. 1, pp. 1-3, Jan. 1958.
[2] R.M. Chavez and G.F. Cooper,"An empirical evaluation of a randomized algorithm for probabilistic inference," Uncertainty in Artificial Intelligence 5, M. Henrion, R.D. Shachter, L. Kanal, and J.F. Lemmer, eds., NorthHolland, Amsterdam, pp. 191-208, 1990.
[3] C.K. Chow and C.N. Liu,"Approximating discrete probability distributions with dependence trees," IEEE Trans. Information Theory, vol. 14, no. 3, pp. 462-467, May 1968.
[4] G.F. Cooper,"NESTOR: A computer-based medical diagnostic aid that integrates causal and probabilistic knowledge," PhD dissertation, Stanford Univ., Palo Alto, Calif., 1984.
[5] G.F. Cooper,"The computational complexity of probabilistic inference using Bayesian belief networks," Artificial Intelligence, vol. 42, pp. 393-405, 1990.
[6] G.F. Cooper and E. Herskovitz,"A Bayesian method for constructing Bayesian belief networks from databases," Proc. 7th Ann. Conf. Uncertainty in Artificial Intelligence, pp. 86-94, 1991.
[7] A.P. Dawid,"The well-calibrated Bayesian," J. American Statistical Assoc., vol. 77, pp. 605-613, 1982.
[8] R.O. Duda,P.E. Hart,K. Konolige,, and R. Reboh,"A computer-based consultant for mineral exploration," Final Report, SRI Projects 6415, SRI International, Menlo Park, Calif., Sept. 1979.
[9] R. Fung and K. Chang,"Weighing and integrating evidence for stochastic simulation in bayesian networks," Uncertainty in Artificial Intelligence 5, M. Henrion, R.D. Shachter, L. Kanal, and J.F. Lemmer, eds., NorthHolland, Amsterdam, pp. 209-220, 1990.
[10] I.J. Good,"Rational Decisions," J. Royal Statistical Society, Ser. B, vol. 14, pp. 107-114, 1952.
[11] M. Henrion,"Propagating uncertainty in Bayesian networks by probabilistic logic sampling," Uncertainty in Artificial Intelligence 2, J.F. Lemmer and L. Kanal, eds., NorthHolland, Amsterdam, pp. 149-164, 1988.
[12] E. Herskovitz and G.F. Cooper,"Kutato: An entropy-driven system for construction of probabilistic expert systems from databases," Uncertainty in Artificial Intelligence 6, P.P. Bonnisone, M. Henrion, L.N. Kanal, and J.F. Lemmer, eds., NorthHolland, Amsterdam, pp. 117-125, 1991.
[13] R.A. Howard,"The foundations of decision analysis," IEEE Trans. Systems Science and Cybernetics, vol. 4, pp. 211-219, 1968.
[14] E.T. Jaynes,"On the rationale of maximum entropy methods," Proc. IEEE, vol. 70, no.9, pp. 939-952, 1982.
[15] S. Kullback,Information Theory and Statistics, John Wiley&Sons: New York, 1959.
[16] S.L. Lauritzen and D.J. Spiegelhalter,"Local computation with probabilities in graphical structures and their applications to expert systems," J. Royal Statistical Society B, vol. 50, no. 2, pp. 157-224, 1988.
[17] J. Pearl,"Reverend Bayes on inference engines: A distributed hierarchical approach," Proc. Nat'l Conf. in AI,Pittsburgh, pp. 133-136, 1982.
[18] J. Pearl,"Fusion, propagation, and structuring in belief networks," Artificial Intelligence, vol. 29, pp. 241-288, 1986.
[19] J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Mateo, Calif.: Morgan Kaufman, 1988.
[20] M.J. Prietula and H.A. Simon,"The experts in your midst," Harvard Business Review, pp. 120-124, Jan.-Feb. 1989.
[21] J.R. Quinlan,"Induction of decision trees," Machine Learning, vol. 1, pp. 81-106, 1986.
[22] H.A. Raiffa and R. Schlaifer,Applied Statistical Decision Theory,Cambridge: MIT Press, 1968.
[23] G. Rebane and J. Pearl,"The recovery of causal polytrees from statistical data," Proc. Third Workshop Uncertainty in AI,Seattle, pp. 222-228, 1987.
[24] R.W. Robinson,"Counting unlabeled acyclic digraphs," Proc. Fifth Australian Conf. Combinatorial Math.,Melbourne, Australia, pp. 28-43, 1976.
[25] T.B. Roby,"Belief states and the uses of evidence," Behavioral Science, vol. 10, pp. 255-270, 1965.
[26] S. Sarkar,H. Mendelson,, and V.S. Storey,"Approximate representation of probabilistic data in expert systems," Working Paper, Dept. Quantitative Business Analysis, Louisiana State Univ., Baton Rouge, 1994.
[27] L.J. Savage,"Elicitation of personal probabilities and expectations," J. American Statistical Assoc., vol. 66, no. 336, Dec. 1971.
[28] M.J. Shaw,"Applying inductive learning to enhance knowledge-based expert systems," Decision Support Systems, vol. 3, pp. 319-332, 1987.
[29] R.D. Shachter and M. Peot,"Simulation approaches to general probabilistic inference on belief networks," Uncertainty in Artificial Intelligence 5, M. Henrion, R.D. Shachter, L. Kanal, and J.F. Lemmer, eds., NorthHolland, Amsterdam, pp. 221-231, 1990.
[30] P. Smyth and R. Goodman, "An Information Theoretic Approach to Rule Induction from Databases," IEEE Trans Knowledge and Data Eng., vol. 4, no. 4, pp. 301-316, Aug. 1992.
[31] C.S. Spetzler and C.-A.S. Stael von Holstein,"Probability encoding in decision analysis," Management Science, vol. 22, no. 3, pp. 340-358, 1975.
[32] S. Srinivas,S. Russell,, and A. Agogino,"Automated construction of sparse Bayesian networks from unstructured probabilistic models and domain information," Uncertainty in Artificial Intelligence 5, North-Holland, pp. 295-308, 1990.
[33] C.-A.S. Stael von Holstein,"Assessment and evaluation of subjective probability distributions," The Economic Research Inst. at the Stockholm School of Economics,Stockholm, 1970.
[34] A. Tversky and D. Kahneman,"Judgement under uncertainty: heuristics and biases," Science, vol. 185, pp. 1,124-1,131, 1974.
[35] R. Uthurusamy,U.M. Fayyad, and S. Spangler,"Learning useful rules from inconclusive data," Knowledge Discovery in Databases, G. Piatetsky-Shapiro and W. J. Frawley, eds., pp. 141-157, 1991.

Index Terms:
Belief networks, expert systems, information theory, knowledge acquisition, probabilistic reasoning, scoring rules.
Sumit Sarkar, Ishwar Murthy, "Constructing Efficient Belief Network Structures With Expert Provided Information," IEEE Transactions on Knowledge and Data Engineering, vol. 8, no. 1, pp. 134-143, Feb. 1996, doi:10.1109/69.485642
Usage of this product signifies your acceptance of the Terms of Use.