This Article 
 Bibliographic References 
 Add to: 
Designing Optimal Sequential Experiments for a Bayesian Classifier
March 1999 (vol. 21 no. 3)
pp. 193-201

Abstract—As computing power has grown, the trend in experimental design has been from techniques requiring little computation towards techniques providing better, more general results at the cost of additional computation. This paper continues this trend presenting three new methods for designing experiments. A summary of previous work in experimental design is provided and used to show how these new methods generalize previous criteria and provide a more accurate analysis than prior methods. The first method generates experimental designs by maximizing the uncertainty of the experiment's result, while the remaining two methods minimize an approximation of the variance of a function of the parameters. The third method uses a computationally expensive discrete approximation to determine the variance. The methods are tested and compared using the logistic model and a Bayesian classifier. The results show that at the expense of greater computation, experimental designs more effective at reducing the uncertainty of the decision boundary of the Bayesian classifier can be generated.

[1] E. Bedrick, R. Christensen., and W. Johnson, "Bayesian Methods for Binomial Regression," unpublished preprint, Statistics Dept., Univ. of California, Davis, 1993.
[2] G.E.P. Box and N.R. Draper, Empirical Model Building and Response Surfaces. New York: John Wiley and Sons, 1986.
[3] K. Chaloner and K. Larntz, "Optimal Bayesian Design Applied to Logistic Regression Experiments," J. Statistical Planning and Inference, vol. 21, pp. 191-208, 1989.
[4] H. Chernoff, "Locally Optimal Designs for Estimating Parameters," Annals of Mathematical Statistics, vol. 24, pp. 586-602, 1953.
[5] R. Davis and A. Prieditis, "Generating Optimal Sequential Experiments for a Bayesian Classifier," Research Report CSE-95-12, Univ. of California at Davis, Computer Science Dept., 1995.
[6] M. DeGroot, Probability and Statistics, 2nd ed. Reading, Mass.: Addison-Wesley, 1986.
[7] W. Dixon and A. Mood, "A Method for Obtaining and Analyzing Sensitivity Data," J. Am. Statistical Assoc., vol. 43, pp. 109-126, 1948.
[8] R. Duda and P. Hart, Pattern Classification and Scene Analysis.New York: John Wiley and Sons, 1973.
[9] B. Falkenhainer and S. Rajamoney, "The Interdependencies of Theory Formation, Revision, and Experimentation," Proc. Fifth Int'l Workshop Machine Learning, Univ. of Michigan, Ann Arbor, June 1988.
[10] P. Freeman, "Optimal Bayesian Sequential Estimation of the Median Effective Dose," Biometrika, vol. 57, no. 1, pp. 79-89, 1970.
[11] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[12] D. Lindley, "On a Measure of Information Provided by an Experiment," Annals of Mathematical Statistics, vol. 27, pp. 986-1,005, 1956.
[13] T. Mitchell, "Generalization as Search," Artificial Intelligence, vol. 18, no. 2, pp. 203-226, 1982.
[14] J. Nelder and R. Mead, "A Simplex Method for Function Minimization," Computer J., vol. 7, pp. 308-313, 1971.
[15] W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling, Numerical Recipes in C.Cambridge, England: Cambridge Univ. Press, 1988.
[16] H. Robbins and S. Monro, "A Stochastic Approximation Method," Annals of Mathematic Statistics, vol. 29, pp. 400-407, 1951.
[17] M. Silvapulle, "On the Existence of Maximum Likelihood Estimators for the Binomial Response Model," J. Royal Statistical Soc., Series B, vol. 43, pp. 310-313, 1981.
[18] D. Subramanian, D. Feigenbaum, and J. Feigenbaum, "Factorization in Experiment Generation," Proc. Am. Assoc. for Artificial Intelligence,Philadelphia, Aug. 1986.
[19] R. Tsutakawa, "Design of Experiment for Bioassay," J. Am. Statistical Assoc., vol. 67, pp. 584-590, 1972.
[20] R. Tsutakawa, "Selection of Dose Levels for Estimating a Percentage Point of a Logistic Quantal Response Curve," Applied Statistics, vol. 29, no. 1, pp. 25-33, 1980.
[21] S. Weiss and C. Kulikowski, Computer Systems That Learn.San Mateo, Calif.: Morgan Kaufman Publishers, Inc., 1991.
[22] G. Wetherill, "Design of Experiment for Bioassay," J. Royal Statistical Soc., Series B, vol. 24, pp. 1-48, 1963.
[23] C. Wu, "Efficient Sequential Designs With Binary Data," J. Am. Statistical Assoc., vol. 80, no. 392, pp. 974-984, Dec. 1985.
[24] J. Zytkow, J. Zhu, and A. Hussam, "Automated Discovery in a Chemistry Laboratory," Proc. Am. Assoc. for Artificial Intelligence,Boston, Aug. 1990.

Index Terms:
Machine learning; Bayesian classifiers; experimental design.
Robert Davis, Armand Prieditis, "Designing Optimal Sequential Experiments for a Bayesian Classifier," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 3, pp. 193-201, March 1999, doi:10.1109/34.754585
Usage of this product signifies your acceptance of the Terms of Use.