The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - November (2006 vol.28)
pp: 1809-1817
W.A. Yousef , Center for Devices & Radiol. Health, Food & Drug Adm., Rockville, MD
R.F. Wagner , Center for Devices & Radiol. Health, Food & Drug Adm., Rockville, MD
ABSTRACT
This paper considers binary classification. We assess a classifier in terms of the area under the ROC curve (AUC). We estimate three important parameters, the conditional AUC (conditional on a particular training set) and the mean and variance of this AUC. We derive, as well, a closed form expression of the variance of the estimator of the AUG. This expression exhibits several components of variance that facilitate an understanding for the sources of uncertainty of that estimate. In addition, we estimate this variance, i.e., the variance of the conditional AUC estimator. Our approach is nonparametric and based on general methods from U-statistics; it addresses the case where the data distribution is neither known nor modeled and where there are only two available data sets, the training and testing sets. Finally, we illustrate some simulation results for these estimators
INDEX TERMS
Training data, Parameter estimation, Uncertainty, Testing, Statistical analysis, Statistical distributions, Decision theory, Probability density function, Random variables, Medical diagnosis,ROC analysis., Classification, nonparametric statistics
CITATION
W.A. Yousef, R.F. Wagner, M.H. Loew, "Assessing Classifiers from Two Independent Data Sets Using ROC Analysis: A Nonparametric Approach", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.28, no. 11, pp. 1809-1817, November 2006, doi:10.1109/TPAMI.2006.218
24 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool