This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
{\cal U}Boost: Boosting with the Universum
April 2012 (vol. 34 no. 4)
pp. 825-832
ASCII Text
x 
 
Chunhua Shen, Peng Wang, Fumin Shen, Hanzi Wang, "{\cal U}Boost: Boosting with the Universum," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 4, pp. 825-832, April, 2012.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2011.240,
author = {Chunhua Shen and Peng Wang and Fumin Shen and Hanzi Wang},
title = {{\cal U}Boost: Boosting with the Universum},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {34},
number = {4},
issn = {0162-8828},
year = {2012},
pages = {825-832},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2011.240},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - {\cal U}Boost: Boosting with the Universum
IS - 4
SN - 0162-8828
SP825
EP832
EPD - 825-832
A1 - Chunhua Shen,
A1 - Peng Wang,
A1 - Fumin Shen,
A1 - Hanzi Wang,
PY - 2012
KW - Universum
KW - kernel methods
KW - boosting
KW - column generation
KW - convex optimization.
VL - 34
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Chunhua Shen, The University of Adelaide, Adelaide
Peng Wang, Beihang University, Beijing
Fumin Shen, Nanjing University of Science and Technology, Nanjing
Hanzi Wang, Xiamen University, Xiamen
It has been shown that the Universum data, which do not belong to either class of the classification problem of interest, may contain useful prior domain knowledge for training a classifier [1], [2]. In this work, we design a novel boosting algorithm that takes advantage of the available Universum data, hence the name {\cal U}Boost. {\cal U}Boost is a boosting implementation of Vapnik's alternative capacity concept to the large margin approach. In addition to the standard regularization term, {\cal U}Boost also controls the learned model's capacity by maximizing the number of observed contradictions. Our experiments demonstrate that {\cal U}Boost can deliver improved classification accuracy over standard boosting algorithms that use labeled data alone.

[1] V. Vapnik, Estimation of Dependences Based on Empirical Data. Springer-Verlag, 2006.
[2] J. Weston, R. Collobert, F. Sinz, L. Bottou, and V. Vapnik, "Inference with the Universum," Proc. Int'l Conf. Machine Learning, pp. 1009-1016, 2006.
[3] F.H. Sinz, O. Chapelle, A. Agarwal, and B. Schölkopf, "An Analysis of Inference with the Universum," Advances in Neural Information Processing Systems, pp. 1369-1376, MIT Press, 2007.
[4] D. Zhang, J. Wang, F. Wang, and C. Zhang, "Semi-Supervised Classification with the Universum," Proc. SIAM Int'l Conf. Data Mining, pp. 323-333, 2008.
[5] B. Peng, G. Qian, and Y. Ma, "View-Invariant Pose Recognition Using Multilinear Analysis and the Universum," Proc. Int'l Symp. Visual Computing, vol. 5359, pp. 581-591, 2008.
[6] B. Peng, G. Qian, and Y. Ma, "Recognizing Body Poses Using Multilinear Analysis and Semi-Supervised Learning," Pattern Recognition Letters, vol. 30, no. 14, pp. 1289-1294, 2009.
[7] X. Bai and V. Cherkassky, "Gender Classification of Human Faces Using Inference through Contradictions," Proc. IEEE Int'l Joint Conf. Neural Networks, pp. 746-750, 2008.
[8] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, 2004.
[9] R.E. Schapire and Y. Freund, "Improved Boosting Algorithms Using Confidence-Rated Predictions," Machine Learning, vol. 37, no. 3, pp. 297-336, Dec. 1999.
[10] C. Shen and H. Li, "On the Dual Formulation of Boosting Algorithms," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 12, pp. 2216-2231, Dec. 2010.
[11] R. Meir and G. Rätsch, "An Introduction to Boosting and Leveraging," Advanced Lectures on Machine Learning, pp. 118-183, Springer-Verlag, 2003.
[12] C. Shen, P. Wang, and H. Li, "LACBoost and FisherBoost: Optimally Building Cascade Classifiers," Proc. European Conf. Computer Vision, vol. 2, pp. 608-621, 2010.
[13] J.H. Friedman, "Greedy Function Approximation: A Gradient Boosting Machine," Annals of Statistics, vol. 29, no. 5, pp. 1189-1232, 2001.
[14] C. Shen and H. Li, "Boosting through Optimization of Margin Distributions," IEEE Trans. Neural Networks, vol. 21, no. 4, pp. 659-666, Apr. 2010.
[15] A. Demiriz, K. Bennett, and J. Shawe-Taylor, "Linear Programming Boosting via Column Generation," Machine Learning, vol. 46, no. 1-3, pp. 225-254, 2002.
[16] L. Mason, J. Baxter, P. Bartlett, and M. Frean, "Boosting Algorithms as Gradient Descent," Advances in Neural Information Processing Systems, pp. 512-518, MIT Press, 2000.
[17] M. Collins, R.E. Schapire, and Y. Singer, "Logistic Regression, AdaBoost and Bregman Distances," Machine Learning, vol. 48, nos. 1-3, pp. 253-285, 2002.
[18] C. Zhu, R.H. Byrd, and J. Nocedal, "L-BFGS-B: Algorithm 778: L-BFGS-B, FORTRAN Routines for Large Scale Bound Constrained Optimization," ACM Trans. Math. Software, vol. 23, no. 4, pp. 550-560, 1997.
[19] S. Maji and J. Malik, "Fast and Accurate Digit Classification," Technical Report UCB/EECS-2009-159, EECS Dept., Univ. of California, Berkeley, Nov. 2009.
[20] C. Schüldt, I. Laptev, and B. Caputo, "Recognizing Human Actions: A Local SVM Approach," Proc. Int'l Conf. Pattern Recognition, vol. 3, pp. 32-36, 2004.
[21] I. Laptev, "On Space-Time Interest Points," Int'l J. Computer Vision, vol. 64, nos. 2-3, pp. 107-123, Sept. 2005.
[22] N. Slonim and N. Tishby, "Agglomerative Information Bottleneck," Advances in Neural Information Processing Systems, vol. 12, pp. 617-623, MIT Press, 1999.

Index Terms:
Universum, kernel methods, boosting, column generation, convex optimization.
Citation:
Chunhua Shen, Peng Wang, Fumin Shen, Hanzi Wang, "{\cal U}Boost: Boosting with the Universum," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 4, pp. 825-832, April 2012, doi:10.1109/TPAMI.2011.240
Usage of this product signifies your acceptance of the Terms of Use.