Development of a Bayesian Framework for Determining Uncertainty in Receiver Operating Characteristic Curve Estimates
Issue No. 01 - January (2010 vol. 22)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TKDE.2009.50
David R. Parker , Pacific Air Forces, Wright-Patterson AFB
Steven C. Gustafson , Air Force Institute of Technology, Wright-Patterson AFB
Mark E. Oxley , Air Force Institute of Technology, Wright-Patterson AFB
Timothy D. Ross , Air Force Research Laboratory Sensors Directorate, AFRL COMPASE Center, Wright-Patterson AFB
This research uses a Bayesian framework to develop probability densities for the receiver operating characteristic (ROC) curve. The ROC curve is a discrimination metric that may be used to quantify how well a detection system classifies targets and nontargets. The degree of uncertainty in ROC curve formulation is a concern that previous research has not adequately addressed. This research formulates a probability density for the ROC curve and characterizes its uncertainty using confidence bands. Methods for the generation and characterization of the probability densities of the ROC curve are specified and demonstrated, where the initial analysis employs beta densities to model target and nontarget samples of detection system output. For given target and nontarget data, given functional forms of the data densities (such as beta density forms) and given prior densities of the form parameters, the methods developed here provide exact performance metric probability densities.
Performance evaluation, performance metrics, receiver operating characteristic, ROC curves, uncertainty estimation, target detection.
M. E. Oxley, T. D. Ross, D. R. Parker and S. C. Gustafson, "Development of a Bayesian Framework for Determining Uncertainty in Receiver Operating Characteristic Curve Estimates," in IEEE Transactions on Knowledge & Data Engineering, vol. 22, no. , pp. 31-45, 2009.