A Mixture Model and EM-Based Algorithm for Class Discovery, Robust Classification, and Outlier Rejection in Mixed Labeled/Unlabeled Data Sets
Issue No. 11 - November (2003 vol. 25)
David J. Miller , IEEE
<p><b>Abstract</b>—Several authors have shown that, when labeled data are scarce, improved classifiers can be built by augmenting the training set with a large set of unlabeled examples and then performing suitable learning. These works assume each unlabeled sample originates from one of the (known) classes. Here, we assume each unlabeled sample comes from either a known <it>or</it> from a heretofore <it>undiscovered</it> class. We propose a novel mixture model which treats as observed data not only the feature vector and the class label, but also the <it>fact</it> of label presence/absence for each sample. Two types of mixture components are posited. "Predefined" components generate data from known classes and assume class labels are <it>missing at random</it>. "Nonpredefined" components only generate unlabeled data—i.e., they capture <it>exclusively unlabeled</it> subsets, consistent with an outlier distribution or new classes. The predefined/nonpredefined natures are <it>data-driven</it>, learned along with the other parameters via an extension of the EM algorithm. Our modeling framework addresses problems involving both the known and unknown classes: 1) robust classifier design, 2) classification with rejections, and 3) identification of the unlabeled samples (and their components) from unknown classes. Case 3 is a step toward new class discovery. Experiments are reported for each application, including topic discovery for the <it>Reuters</it> domain. Experiments also demonstrate the value of label presence/absence data in learning accurate mixtures. </p>
Class discovery, labeled and unlabeled data, outlier detection, sample rejection, mixture models, EM algorithm, text categorization.
David J. Miller, John Browning, "A Mixture Model and EM-Based Algorithm for Class Discovery, Robust Classification, and Outlier Rejection in Mixed Labeled/Unlabeled Data Sets", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 25, no. , pp. 1468-1483, November 2003, doi:10.1109/TPAMI.2003.1240120