Issue No.05 - May (2010 vol.22)
Leandro L. Minku , University of Birmingham, Birmingham
Allan P. White , University of Birmingham, Birmingham
Xin Yao , University of Birmingham, Birmingham
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TKDE.2009.156
Online learning algorithms often have to operate in the presence of concept drift (i.e., the concepts to be learned can change with time). This paper presents a new categorization for concept drift, separating drifts according to different criteria into mutually exclusive and nonheterogeneous categories. Moreover, although ensembles of learning machines have been used to learn in the presence of concept drift, there has been no deep study of why they can be helpful for that and which of their features can contribute or not for that. As diversity is one of these features, we present a diversity analysis in the presence of different types of drifts. We show that, before the drift, ensembles with less diversity obtain lower test errors. On the other hand, it is a good strategy to maintain highly diverse ensembles to obtain lower test errors shortly after the drift independent on the type of drift, even though high diversity is more important for more severe drifts. Longer after the drift, high diversity becomes less important. Diversity by itself can help to reduce the initial increase in error caused by a drift, but does not provide the faster recovery from drifts in long-term.
Concept drift, online learning, neural network ensembles, diversity.
Leandro L. Minku, Allan P. White, Xin Yao, "The Impact of Diversity on Online Ensemble Learning in the Presence of Concept Drift", IEEE Transactions on Knowledge & Data Engineering, vol.22, no. 5, pp. 730-742, May 2010, doi:10.1109/TKDE.2009.156