This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Bagging with Adaptive Costs
May 2008 (vol. 20 no. 5)
pp. 577-588
ASCII Text
x 
 
Yi Zhang, W. Nick Street, "Bagging with Adaptive Costs," IEEE Transactions on Knowledge and Data Engineering, vol. 20, no. 5, pp. 577-588, May, 2008.
 
 
BibTex
x
 
@article{ 10.1109/TKDE.2007.190724,
author = {Yi Zhang and W. Nick Street},
title = {Bagging with Adaptive Costs},
journal ={IEEE Transactions on Knowledge and Data Engineering},
volume = {20},
number = {5},
issn = {1041-4347},
year = {2008},
pages = {577-588},
doi = {http://doi.ieeecomputersociety.org/10.1109/TKDE.2007.190724},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Knowledge and Data Engineering
TI - Bagging with Adaptive Costs
IS - 5
SN - 1041-4347
SP577
EP588
EPD - 577-588
A1 - Yi Zhang,
A1 - W. Nick Street,
PY - 2008
KW - Mining methods and algorithms
KW - Data mining
VL - 20
JA - IEEE Transactions on Knowledge and Data Engineering
ER -
 
Ensemble methods have proved to be highly effective in improving the performance of base learners under most circumstances. In this paper, we propose a new algorithm that combines the merits of some existing techniques, namely bagging, arcing and stacking. The basic structure of the algorithm resembles bagging. However, the misclassification cost of each training point is repeatedly adjusted according to its observed out-of-bag vote margin. In this way, the method gains the advantage of arcing - building the classifier the ensemble needs - without fixating on potentially noisy points. Computational experiments show that this algorithm performs consistently better than bagging and arcing with linear and nonlinear base classifiers. In view of the characteristics of bacing, a hybrid ensemble learning strategy, which combines bagging and different versions of bacing, is proposed and studied empirically.

[1] A. Krogh and J. Vedelsby, “Neural Network Ensembles, Cross Validation, and Active Learning,” Advances in Neural Information Processing Systems, G. Tesauro, D. Touretzky, and T. Leen, eds., vol. 7, MIT Press, pp. 231-238, 1995, citeseer.nj.nec.comkrogh95neural.html.
[2] T.G. Dietterich, “Ensemble Methods in Machine Learning,” Lecture Notes in Computer Science 1857, pp. 1-15, 2000, citeseer.nj. nec.comdietterich00ensemble.html .
[3] E. Bauer and R. Kohavi, “An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants,” Machine Learning, vol. 36, no. 1-2, pp. 105-139, 1999, citeseer. ist.psu. edubauer99empirical.html .
[4] D. Opitz and R. Maclin, “Popular Ensemble Methods: An Empirical Study,” J. Artificial Intelligence Research, vol. 11, pp.169-198, 1999, citeseer.ist.psu.eduopitz99popular.html.
[5] T.G. Dietterich, “An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization,” Machine Learning, vol. 40, no. 2, pp. 139-157, 2000.
[6] L. Breiman, “Bagging Predictors,” Machine Learning, vol. 24, no. 2, pp. 123-140, 1996, citeseer.nj.nec.combreiman96bagging.html .
[7] Y. Freund and R.E. Schapire, “Experiments with a New Boosting Algorithm,” Proc. 13th Int'l Conf. Machine Learning, pp. 148-156, 1996, citeseer.nj.nec.comfreund96experiments.html .
[8] L. Breiman, “Arcing Classifiers,” Annals of Statistics, vol. 26, pp.801-849, 1998.
[9] D.H. Wolpert, “Stacked Generalization,” Neural Networks, vol. 5, pp. 241-259, 1992, .
[10] L. Breiman, “Random Forests,” Machine Learning, vol. 45, no. 1, pp.5-32, 2001, .
[11] A.J. Grove and D. Schuurmans, “Boosting in the Limit: Maximizing the Margin of Learned Ensembles,” Proc. Conf Innovative Applications of AI (AAAI/IAAI '98), pp. 692-699, 1998, citeseer.nj.nec.com/wolpert92stacked.htmlciteseer.nj.nec.com/ breiman01random.htmlciteseer. ist.psu.edu grove98boosting.html .
[12] G. Ridgeway, “Discussion of Additive Logistic Regression: A Statistical View of Boosting,” Annals of Statistics, vol. 28, pp. 393-400, 2000.
[13] P.S. Bradley, U.M. Fayyad, and O.L. Mangasarian, “Mathematical Programming for Data Mining: Formulations and Challenges,” INFORMS J. Computing, vol. 11, no. 3, pp. 217-238, 1999, citeseer.nj. nec.combradley98mathematical.html .
[14] V.N. Vapnik, The Nature of Statistical Learning Theory. Springer, 1995.
[15] C. Blake and C. Merz UCI Repository of Machine Learning Databases, http://www.ics.uci.edu/~mlearnMLRepository. html , 1998.
[16] Y. Freund, and R.E. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” Proc. European Conf. Computational Learning Theory, pp. 23-37, 1995, citeseer.nj.nec.com/articlefreund95decisiontheoretic.html .
[17] S. Abney, R. Schapire, and Y. Singer, “Boosting Applied to Tagging and PP Attachment,” Proc. Joint SIGDAT Conf. Empirical Methods in Natural Language Processing and Very Large Corpora, 1999, citeseer.nj.nec.comabney99boosting.html.
[18] R.J. Quinlan, C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.

Index Terms:
Mining methods and algorithms, Data mining
Citation:
Yi Zhang, W. Nick Street, "Bagging with Adaptive Costs," IEEE Transactions on Knowledge and Data Engineering, vol. 20, no. 5, pp. 577-588, May 2008, doi:10.1109/TKDE.2007.190724
Usage of this product signifies your acceptance of the Terms of Use.