The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.06 - June (2009 vol.31)
pp: 1140-1146
Oriol Pujol , Universitat de Barcelona, Barcelona
David Masip , Universitat Oberta de Catalunya, Barcelona
ABSTRACT
This article introduces a novel binary discriminative learning technique based on the approximation of the non-linear decision boundary by a piece-wise linear smooth additive model. The decision border is geometrically defined by means of the characterizing boundary points - points that belong to the optimal boundary under a certain notion of robustness. Based on these points, a set of locally robust linear classifiers is defined and assembled by means of a Tikhonov regularized optimization procedure in an additive model to create a final lambda-smooth decision rule. As a result, a very simple and robust classifier with a strong geometrical meaning and non-linear behavior is obtained. The simplicity of the method allows its extension to cope with some of nowadays machine learning challenges, such as online learning, large scale learning or parallelization, with linear computational complexity. We validate our approach on the UCI database. Finally, we apply our technique in online and large scale scenarios, and in six real life computer vision and pattern recognition problems: gender recognition, intravascular ultrasound tissue classification, speed traffic sign detection, Chagas' disease severity detection, clef classification and action recognition using a 3D accelerometer data. The results are promising and this paper opens a line of research that deserves further attention.
INDEX TERMS
Machine learning, Computer vision
CITATION
Oriol Pujol, David Masip, "Geometry-Based Ensembles: Toward a Structural Characterization of the Classification Boundary", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 6, pp. 1140-1146, June 2009, doi:10.1109/TPAMI.2009.31
REFERENCES
[1] J. Friedman, T. Hastie, and R. Tibshirani, “Additive Logistic Regression: A Statistical View of Boosting,” Annals of Statistics, pp.337-407, 2000.
[2] V. Vapnik, The Nature of Statistical Learning Theory. Springer, 1995.
[3] B.K. Bhattacharya, R.S. Poulsen, and G.T. Toussaint, “Application of Proximity Graphs to Editing Nearest Neighbor Decision Rule,” Proc. Int'l Symp. Information Theory, 1981.
[4] K. Mukherjee, “Application of the Gabriel Graph to Instance Based Learning,” MSc thesis, Simon Fraser Univ., 2004.
[5] W. Zhang and I. King, “A Study of the Relationship between Support Vector Machine and Gabriel Graph,” Proc. Int'l Conf. Neural Networks, vol. 1, pp.239-244, 2002.
[6] K.R. Gabriel and R.R. Sokal, “A New Statistical Approach to Geographic Variation Analysis,” Systematic Zoology, vol. 18, pp.259-270, 1969.
[7] B. Bhattacharya, K. Mukherjee, and G.T. Toussaint, “Geometric Decision Rules for Instance-Based Learning Algorithms,” Proc. First Int'l Conf. Pattern Recognition and Machine Intelligence, pp.60-69, 2005.
[8] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. Springer, 2001.
[9] P.M. Murphy and D.W. Aha, UCI Repository of Machine Learning Databases, Univ. of California, Dept. of Information and Computer Science, 1994.
[10] A. Martinez and R. Benavente, “The AR Face Database,” Technical Report 24, Computer Vision Center, Spain, 1998.
[11] K. Bennett and E. Bredensteiner, “Duality and Geometry in SVM Classifiers,” Proc. 17th Int'l Conf. Machine Learning, pp.57-64, 2000.
[12] J. Bi and K.P. Bennett, ”A Geometric Approach to Support Vector Regression,” Neurocomputing, vol. 55, pp.79-108, 2003.
[13] S. Escalera, O. Pujol, E. Laciar, J. Vitrià, E. Pueyo, and P. Radeva, “Coronary Damage Classification of Patients with the Chagas' disease with Error-Correcting Output Codes,” Proc. IEEE Int'l Conf. Intelligence Systems, vol. 2, pp.12-17-12-22, 2008.
[14] K.L. Caballero, J. Barajas, O. Pujol, N. Salvatella, and P. Radeva, “In-Vivo IVUS Tissue Classification: A Comparison between RF Signal Analysis and Reconstructed Images,” Lecture Notes in Computer Science, vol. 4225, pp.137-146, 2006.
[15] M. Bern, D. Eppstein, and F. Yao, “The Expected Extremes in a Delaunay Triangulation,” Int'l J. Computational Geometry and Applications, vol. 1, no. 1, pp.79-92, 1991.
[16] C. Bishop and M. Tipping, “Variational Relevance Vector Machines,” Proc. 16th Conf. Uncertainty in Artificial Intelligence, pp.46-53, 2000.
[17] O. Hamsici and A. Martinez, “Spherical-Homoscedastic Distributions: The Equivalency of Spherical and Normal Distributions in Classification,” J.Machine Learning Research, vol. 8, pp.1583-1623, 2007.
[18] J. Demsar, “Statistical Comparison of Classifiers over Multiple Data Sets,” J.Machine Learning Research, vol. 7, pp.1-30, 2006.
[19] S. Escalera, A. Fornes, O. Pujol, J. Llados, and P. Radeva, “Multi-Class Binary Object Categorization Using Blurred Shape Models,” Proc. Iberoam. Congress Pattern Recognition, pp.142-151, 2007.
[20] D. Lewis, Y. Yang, T. Rose, and F. Li, “RCV1: A New Benchmark Collection for Text Categorization Research,” J. Machine Learning Research, vol. 5. pp.361-397, 2004.
[21] OSU SVM Toolbox, http:/svm.sourceforge.net, 2009.
[22] PRTools Toolbox, Faculty of Applied Physics, Delft Univ. of Technology, http:/ww.prtools.org, 2009.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool