
This Article  
 
Share  
Bibliographic References  
Add to:  
Digg Furl Spurl Blink Simpy Del.icio.us Y!MyWeb  
Search  
 
ASCII Text  x  
David R. Parker, Steven C. Gustafson, Mark E. Oxley, Timothy D. Ross, "Development of a Bayesian Framework for Determining Uncertainty in Receiver Operating Characteristic Curve Estimates," IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 1, pp. 3145, January, 2010.  
BibTex  x  
@article{ 10.1109/TKDE.2009.50, author = {David R. Parker and Steven C. Gustafson and Mark E. Oxley and Timothy D. Ross}, title = {Development of a Bayesian Framework for Determining Uncertainty in Receiver Operating Characteristic Curve Estimates}, journal ={IEEE Transactions on Knowledge and Data Engineering}, volume = {22}, number = {1}, issn = {10414347}, year = {2010}, pages = {3145}, doi = {http://doi.ieeecomputersociety.org/10.1109/TKDE.2009.50}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, }  
RefWorks Procite/RefMan/Endnote  x  
TY  JOUR JO  IEEE Transactions on Knowledge and Data Engineering TI  Development of a Bayesian Framework for Determining Uncertainty in Receiver Operating Characteristic Curve Estimates IS  1 SN  10414347 SP31 EP45 EPD  3145 A1  David R. Parker, A1  Steven C. Gustafson, A1  Mark E. Oxley, A1  Timothy D. Ross, PY  2010 KW  Performance evaluation KW  performance metrics KW  receiver operating characteristic KW  ROC curves KW  uncertainty estimation KW  target detection. VL  22 JA  IEEE Transactions on Knowledge and Data Engineering ER   
[1] T. Fawcett, “An Introduction to ROC Analysis,” Pattern Recognition Letters, vol. 27, no. 8, pp. 861874, 2006.
[2] J. Swets, “Measuring the Accuracy of Diagnostic Systems,” Science, vol. 240, pp. 12851293, 1988.
[3] T. Wickens, Elementary Signal Detection Theory. Oxford Univ. Press, 2002.
[4] J. Hanley, “Receiver Operating Characteristic (ROC) Curves,” Encyclopedia of Biostatistics, P. Armitage and T. Colton, eds., pp.37383745, Wiley, 1999.
[5] S. Macskassy and F. Provost, “Confidence Bands for ROC Curves: Methods and an Empirical Study,” Proc. First Workshop ROC Analysis in AI (ROCAI '04) at ECAI2004, 2004.
[6] S. Alsing, “The Evaluation of Competing Classifiers,” PhD dissertation, Air Force Inst. of Tech nology, 2000.
[7] T. Ross and M. Minardi, “Discrimination and Confidence Error in Detector Reported Scores,” Proc. SPIE Conf. Algorithms for Synthetic Aperture Radar Imagery XI, pp. 342353, 2004.
[8] D. Parker, S. Gustafson, and T. Ross, “Probability Densities and Confidence Intervals for Target Recognition Performance Metrics,” Proc. SPIE Conf. Algorithms for Synthetic Aperture Radar Imagery XII, pp. 373382, May 2005.
[9] J. Olmstead, Advanced Calculus. AppletonCenturyCrofts, 1961.
[10] C. Metz, B. Herman, and J. Shen, “Maximum Likelihood Estimation of Receiver Operating Characteristic (ROC) Curves from ContinuouslyDistributed Data,” Statistics in Medicine, vol. 17, no. 9, pp. 10331053, 1998.
[11] D. Dorfman, K. Berbaum, C. Metz, R. Lenth, J. Hanley, and H. Dagga, “Proper Receiver Operating Characteristic Analysis: The Bigamma Model,” Academic Radiology, vol. 4, no. 2, pp. 138149, Feb. 1997.
[12] N. Obuchowski and M. Lieber, “Confidence Intervals for the Receiver Operating Characteristic Area in Studies with Small Samples,” Academic Radiology, vol. 5, pp. 561571, 1998.
[13] G. Ma and W. Hall, “Confidence Bands for Receiver Operating Characteristic Curves,” Medical Decision Making, vol. 13, pp. 191197, 1993.
[14] G. Campbell, “Advances in Statistical Methodology for the Evaluation of Diagnostic and Laboratory Tests,” Statistics in Medicine, vol. 13, pp. 499508, 1994.
[15] A. Garber, R. Olshen, H. Zhang, and E. Venkatraman, “Predicting HighRisk Cholesterol Levels,” Int'l Statistical Rev., vol. 62, pp. 203228, 1994.
[16] K. Jensen, H.H. Muller, and H. Schafer, “Regional Confidence Bands for ROC Curves,” Statistics in Medicine, vol. 19, pp. 493509, 2000.
[17] D. Mossman, “Resampling Techniques in the Analysis of NonBinormal ROC Data,” Medical Decision Making, vol. 15, pp. 358366, 1995.
[18] R. Platt, J. Hanley, and H. Yang, “Bootstrap Confidence Intervals for the Sensitivity of a Quantitative Diagnostic Test,” Statistics in Medicine, vol. 19, pp. 313322, 2000.
[19] E. Simpson, R. Ideker, K. Lee, and W. Smith, “Computing ROC Curve Confidence Intervals for Cardiac Activation Detectors,” Proc. 11th Ann. Int'l Conf. IEEE Eng. Medicine and Biology Soc., 1989.
[20] X.H. Zhou and G. Qin, “Improved Confidence Intervals for the Sensitivity at a Fixed Level of Specificity of a ContinuousScale Diagnostic Test,” Statistics in Medicine, vol. 24, pp. 465477, 2005.
[21] B. Efron and R. Tibshirani, An Introduction to the Bootstrap. Chapman and Hall, 1993.
[22] D. Parker, “Uncertainty Estimation for Target Detection System Discrimination and Confidence Performance Metrics,” PhD dissertation, Air Force Inst. of Tech nology, 2006.
[23] C. Lloyd, “Estimation of a Convex ROC Curve,” Statistics and Probability Letters, vol. 59, pp. 99111, 2002.
[24] P. Qiu and C. Le, “ROC Curve Estimation Based on Local Smoothing,” J. Statistical Computation and Simulation, vol. 70, pp. 5569, 2001.
[25] S. Macskassy, F. Provost, and S. Rosset, “Pointwise ROC Confidence Bounds: An Empirical Evaluation,” Proc. Int'l Conf. Machine Learning (ICML '05) Workshop ROC Analysis in Machine Learning, 2005.
[26] R. Hilgers, “DistributionFree ConfidenceBounds for ROC Curves,” Methods of Information in Medicine, vol. 30, no. 2, pp. 96101, Apr. 1991.
[27] J. Kerekes, “Receiver Operating Characteristic Curve Confidence Intervals and Regions,” IEEE Geoscience and Remote Sensing Letters, vol. 5, no. 2, pp. 251255, 2008.
[28] P. Hall, R. Hyndman, and Y. Fan, “Nonparametric Confidence Intervals for Receiver Operating Characteristic Curves,” Biometrika, vol. 91, pp. 743750, 2004.
[29] J. Carlin, “MetaAnalysis for 2 × 2 Tables: A Bayesian Approach,” Statistics in Medicine, vol. 11, pp. 141158, 1992.
[30] V. Dukic and C. Gatsonis, “MetaAnalysis of Diagnostic Test Accuracy Assessment Studies with Varying Number of Thresholds,” Biometrics, vol. 49, pp. 936946, 2003.
[31] M. Hellmich, K. Abrams, and A. Sutton, “Bayesian Approaches to MetaAnalysis of ROC Curves,” Medical Decision Making, vol. 19, pp. 252264, 1999.
[32] C. Rutter and C. Gatsonis, “A Hierarchical Regression Approach to MetaAnalysis of Diagnostic Test Accuracy Evaluations,” Statistics in Medicine, vol. 20, pp. 28652884, 2001.
[33] T. Smith, D. Spiegelhalter, and A. Thomas, “Bayesian Approaches to RandomEffects MetaAnalysis: A Comparative Study,” Statistics in Medicine, vol. 14, pp. 26852699, 1995.
[34] X. Zhou, “Empirical Bayes Combination of Estimated Areas under ROC Curves Using Estimating Equations,” Medical Decision Making, vol. 16, pp. 2428, 1996.
[35] L. Broemeling, “The Predictive Distribution and Area under the ROC Curve,” Technical Report 01304, The Univ. of Texas M.D. Anderson Cancer Center, 2004.
[36] D. Parker, S. Gustafson, and T. Ross, “Bayesian Confidence Intervals for ROC Curves,” IEE Electronics Letters, vol. 41, pp. 279280, 2005.
[37] B. Carlin and T. Louis, Bayes and Empirical Bayes Methods for Data Analysis. Chapman and Hall/CRC, 2000.
[38] D. MacKay, Information Theory, Inference, and Learning Algorithms. Cambridge Univ. Press, 2003.
[39] W. Bolstad, Introduction to Bayesian Statistics. Wiley, 2004.
[40] W. Mendenhall, D. Wackerly, and R. Scheaffer, Mathematical Statistics with Applications. PWSKent, 1990.
[41] J. Patel, C. Kapadia, and D. Owen, Handbook of Statistical Distributions. Marcel Dekker, 1976.
[42] A. Kagan, I. Linnik, and C. Rao, Characterization Problems in Mathematical Statistics. Wiley, 1973.
[43] G. Hahn and S. Shapiro, Statistical Models in Engineering. Wiley, 1967.
[44] D. MacKay, “Bayesian Methods for Adaptive Models,” PhD dissertation, California Inst. of Tech nology, 1992.
[45] D. MacKay, “Bayesian Interpolation,” Neural Computation, vol. 4, pp. 415447, 1992.
[46] C. Bishop, Neural Networks for Pattern Recognition. Oxford Univ. Press, 1995.
[47] M. Clyde, “Bayesian Model Averaging and Model Search Strategies,” Bayesian Statistics 6, J. Berger, A. Dawid, and A.Smith, eds., pp. 157185, Oxford Univ. Press, 1999.
[48] M. Clyde and E. George, “Model Uncertainty,” Statistical Science, vol. 19, pp. 8194, 2004.
[49] J. Hoeting, D. Madigan, A. Raftery, and C. Volin, “Bayesian Model Averaging: A Tutorial,” Statistical Science, vol. 14, no. 4, pp. 382417, http://www.stat.washington.edu/www/research/ onlinehoeting1999.pdf., 1999.
[50] M. Jordan, Z. Ghahramani, T. Jaakkola, and L. Saul, Learning in Graphical Models, pp. 105161. MIT Press, 1999.
[51] R. Larson, R. Hostetler, B. Edwards, and D. Heyd, Calculus with Analytic Geometry, seventh ed. Houghton Mifflin, 2002.
[52] A. Gelman, J. Carlin, H. Stern, and D. Rubin, Bayesian Data Analysis, second ed. Chapman and Hall, 2004.
[53] J. Hammersley and D. Handscomb, Monte Carlo Methods. Methuen, 1964.
[54] R. Kass and A. Raftery, “Bayes Factors,” J. Am. Statistical Assoc., vol. 90, pp. 773795, 1995.
[55] G. Casella and R. Berger, Statistical Inference, second ed. Duxbury, 2002.
[56] P. Gregory, Bayesian Logical Data Analysis for the Physical Sciences: A Comparative Approach with Mathematica Support. Cambridge, 2005.
[57] H. Schafer, “Efficient Confidence Bounds for ROC Curves,” Statistics in Medicine, vol. 13, no. 15, pp. 15511561, 1994.
[58] J. Tilbury, P. VanEetvelt, J. Garibaldi, J. Curnow, and E. Ifeachor, “Receiver Operating Characteristic Analysis for Intelligent Medical Systems—a New Approach for Finding Confidence Intervals,” IEEE Trans. Biomedical Eng., vol. 47, no. 7, pp. 952963, July 2000.
[59] J. Tilbury, “Evaluation of Intelligent Medical Systems,” PhD dissertation, Univ. of Plymouth, England, 2002.
[60] J. Tilbury, P.V. Eetvelt, J. Curnow, and E. Ifeachor, “Objective Evaluation of Intelligent Medical Systems Using a Bayesian Approach to Analysis of ROC Curves,” Proc. Fifth Int'l Conf. Neural Networks and Expert Systems in Medicine and Healthcare/Proc. First Int'l Conf. Computational Intelligence in Medicine and Healthcare, 2003.
[61] C. Bos, “A Comparison of Marginal Likelihood Computation Methods,” Tinbergen Inst. Discussion Paper, vol. 02084/4, 2002.
[62] D. Madigan and A. Raftery, “Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window,” J. Am. Statistical Assoc., vol. 89, no. 428, pp. 15351546, 1994.
[63] A. Raftery, F. Balabdaoui, T. Gneiting, and M. Polakowski, “Using Bayesian Model Averaging to Calibrate Forecast Ensembles,” Technical Report 440, Univ. of Washington, Dec. 2003.
[64] K. Zou, W.W. III, R. Kikinis, and S. Warfield, “Three Validation Metrics for Automated Probabilistic Image Segmentation of Brain Tumors,” Statistics in Medicine, vol. 23, pp. 12591282, 2004.
[65] L. Dodd and M. Pepe, “Partial AUC Estimation and Regression,” Biometrics, vol. 59, pp. 614623, 2003.
[66] D. Faraggi, “Adjusting Receiver Operating Characteristic Curves and Related Indices for Covariates,” Statistician, vol. 51, pp. 179192, 2003.