The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - November (2006 vol.28)
pp: 1809-1817
ABSTRACT
This paper considers binary classification. We assess a classifier in terms of the Area Under the ROC Curve (AUC). We estimate three important parameters, the conditional AUC (conditional on a particular training set) and the mean and variance of this AUC. We derive, as well, a closed form expression of the variance of the estimator of the AUC. This expression exhibits several components of variance that facilitate an understanding for the sources of uncertainty of that estimate. In addition, we estimate this variance, i.e., the variance of the conditional AUC estimator. Our approach is nonparametric and based on general methods from U--statistics; it addresses the case where the data distribution is neither known nor modeled and where there are only two available data sets, the training and testing sets. Finally, we illustrate some simulation results for these estimators.
INDEX TERMS
Classification, nonparametric statistics, ROC analysis.
CITATION
Waleed A. Yousef, Robert F. Wagner, Murray H. Loew, "Assessing Classifiers from Two Independent Data Sets Using ROC Analysis: A Nonparametric Approach", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.28, no. 11, pp. 1809-1817, November 2006, doi:10.1109/TPAMI.2006.218
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool