The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2008 vol.20)
pp: 577-588
ABSTRACT
Ensemble methods have proved to be highly effective in improving the performance of base learners under most circumstances. In this paper, we propose a new algorithm that combines the merits of some existing techniques, namely bagging, arcing and stacking. The basic structure of the algorithm resembles bagging. However, the misclassification cost of each training point is repeatedly adjusted according to its observed out-of-bag vote margin. In this way, the method gains the advantage of arcing - building the classifier the ensemble needs - without fixating on potentially noisy points. Computational experiments show that this algorithm performs consistently better than bagging and arcing with linear and nonlinear base classifiers. In view of the characteristics of bacing, a hybrid ensemble learning strategy, which combines bagging and different versions of bacing, is proposed and studied empirically.
INDEX TERMS
Mining methods and algorithms, Data mining
CITATION
Yi Zhang, W. Nick Street, "Bagging with Adaptive Costs", IEEE Transactions on Knowledge & Data Engineering, vol.20, no. 5, pp. 577-588, May 2008, doi:10.1109/TKDE.2007.190724
REFERENCES
[1] A. Krogh and J. Vedelsby, “Neural Network Ensembles, Cross Validation, and Active Learning,” Advances in Neural Information Processing Systems, G. Tesauro, D. Touretzky, and T. Leen, eds., vol. 7, MIT Press, pp. 231-238, 1995, citeseer.nj.nec.comkrogh95neural.html.
[2] T.G. Dietterich, “Ensemble Methods in Machine Learning,” Lecture Notes in Computer Science 1857, pp. 1-15, 2000, citeseer.nj. nec.comdietterich00ensemble.html .
[3] E. Bauer and R. Kohavi, “An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants,” Machine Learning, vol. 36, no. 1-2, pp. 105-139, 1999, citeseer. ist.psu. edubauer99empirical.html .
[4] D. Opitz and R. Maclin, “Popular Ensemble Methods: An Empirical Study,” J. Artificial Intelligence Research, vol. 11, pp.169-198, 1999, citeseer.ist.psu.eduopitz99popular.html.
[5] T.G. Dietterich, “An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization,” Machine Learning, vol. 40, no. 2, pp. 139-157, 2000.
[6] L. Breiman, “Bagging Predictors,” Machine Learning, vol. 24, no. 2, pp. 123-140, 1996, citeseer.nj.nec.combreiman96bagging.html .
[7] Y. Freund and R.E. Schapire, “Experiments with a New Boosting Algorithm,” Proc. 13th Int'l Conf. Machine Learning, pp. 148-156, 1996, citeseer.nj.nec.comfreund96experiments.html .
[8] L. Breiman, “Arcing Classifiers,” Annals of Statistics, vol. 26, pp.801-849, 1998.
[9] D.H. Wolpert, “Stacked Generalization,” Neural Networks, vol. 5, pp. 241-259, 1992, .
[10] L. Breiman, “Random Forests,” Machine Learning, vol. 45, no. 1, pp.5-32, 2001, .
[11] A.J. Grove and D. Schuurmans, “Boosting in the Limit: Maximizing the Margin of Learned Ensembles,” Proc. Conf Innovative Applications of AI (AAAI/IAAI '98), pp. 692-699, 1998, citeseer.nj.nec.com/wolpert92stacked.htmlciteseer.nj.nec.com/ breiman01random.htmlciteseer. ist.psu.edu grove98boosting.html .
[12] G. Ridgeway, “Discussion of Additive Logistic Regression: A Statistical View of Boosting,” Annals of Statistics, vol. 28, pp. 393-400, 2000.
[13] P.S. Bradley, U.M. Fayyad, and O.L. Mangasarian, “Mathematical Programming for Data Mining: Formulations and Challenges,” INFORMS J. Computing, vol. 11, no. 3, pp. 217-238, 1999, citeseer.nj. nec.combradley98mathematical.html .
[14] V.N. Vapnik, The Nature of Statistical Learning Theory. Springer, 1995.
[15] C. Blake and C. Merz UCI Repository of Machine Learning Databases, http://www.ics.uci.edu/~mlearnMLRepository. html , 1998.
[16] Y. Freund, and R.E. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” Proc. European Conf. Computational Learning Theory, pp. 23-37, 1995, citeseer.nj.nec.com/articlefreund95decisiontheoretic.html .
[17] S. Abney, R. Schapire, and Y. Singer, “Boosting Applied to Tagging and PP Attachment,” Proc. Joint SIGDAT Conf. Empirical Methods in Natural Language Processing and Very Large Corpora, 1999, citeseer.nj.nec.comabney99boosting.html.
[18] R.J. Quinlan, C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
17 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool