2013 IEEE 13th International Conference on Data Mining (2006)
Dec. 18, 2006 to Dec. 22, 2006
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/ICDM.2006.12
Stephen Scott , University of Nebraska, USA
Deng Kun , University of Nebraska, USA
Matt Culver , University of Nebraska, USA
In active learning, a machine learning algorithmis given an unlabeled set of examples U, and is allowed to request labels for a relatively small subset of U to use for training. The goal is then to judiciously choose which examples in U to have labeled in order to optimize some performance criterion, e.g. classification accuracy. We study how active learning affects AUC. We examine two existing algorithms from the literature and present our own active learning algorithms designed to maximize the AUC of the hypothesis. One of our algorithms was consistently the top performer, and Closest Sampling from the literature often came in second behind it. When good posterior probability estimates were available, our heuristics were by far the best.
Stephen Scott, Deng Kun, Matt Culver, "Active Learning to Maximize Area Under the ROC Curve", 2013 IEEE 13th International Conference on Data Mining, vol. 00, no. , pp. 149-158, 2006, doi:10.1109/ICDM.2006.12