This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Multisurface Proximal Support Vector Machine Classification via Generalized Eigenvalues
January 2006 (vol. 28 no. 1)
pp. 69-74
A new approach to support vector machine (SVM) classification is proposed wherein each of two data sets are proximal to one of two distinct planes that are not parallel to each other. Each plane is generated such that it is closest to one of the two data sets and as far as possible from the other data set. Each of the two nonparallel proximal planes is obtained by a single MATLAB command as the eigenvector corresponding to a smallest eigenvalue of a generalized eigenvalue problem. Classification by proximity to two distinct nonlinear surfaces generated by a nonlinear kernel also leads to two simple generalized eigenvalue problems. The effectiveness of the proposed method is demonstrated by tests on simple examples as well as on a number of public data sets. These examples show the advantages of the proposed approach in both computation time and test set correctness.

[1] E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen, LAPACK User's Guide, third ed., Philadelphia: SIAM, 1999, http://www.netlib.orglapack/.
[2] K.P. Bennett and O.L. Mangasarian, “Robust Linear Programming Discrimination of Two Linearly Inseparable Sets,” Optimization Methods and Software, vol. 1, pp. 23-34, 1992.
[3] P.S. Bradley and O.L. Mangasarian, “k-Plane Clustering,” J. Global Optimization, vol. 16, pp. 23-32, 2000, ftp://ftp.cs.wisc.edu/math-prog/tech-reports 98-08.ps.
[4] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge, Mass.: Cambridge Univ. Press, 2000.
[5] J.W. Demmel, Applied Numerical Linear Algebra. Philadelphia: SIAM, 1997.
[6] T. Evgeniou, M. Pontil, and T. Poggio, “Regularization Networks and Support Vector Machines,” Advances in Computational Math., vol. 13, pp. 1-50, 2000.
[7] G. Fung and O.L. Mangasarian, “Proximal Support Vector Machine Classifiers,” Proc. Knowledge Discovery and Data Mining, F. Provost and R. Srikant, eds., pp. 77-86, 2001, ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports 01-02.ps.
[8] G.H. Golub and C.F. Van Loan, Matrix Computations, third ed. Baltimore: The John Hopkins Univ. Press,. 1996.
[9] T. Joachims, “Making Large-Scale Support Vector Machine Learning Practical,” Advances in Kernel Methods— Support Vector Learning, B. Schölkopf, C.J.C. Burges, and A.J. Smola, eds., pp. 169-184, Cambridge, Mass.: MIT Press, 1999.
[10] M. Kojima, S. Mizuno, T. Noma, and A. Yoshise, A Unified Approach to Interior Point Algorithms for Linear Complementarity Problems. Berlin: Springer-Verlag, 1991.
[11] R.H. Leary, J.B. Rosen, and P. Jambeck, “An Optimal Structure-Discrminative Amino Acid Index for Protein Recognition,” Biophysical J., vol. 86, pp. 411-419, 2004.
[12] Y.-J. Lee and O.L. Mangasarian, “RSVM: Reduced Support Vector Machines,” Proc. First SIAM Int'l Conf. Data Mining, Apr. 2001, ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports 00-07.ps.
[13] O.L. Mangasarian, “Least Norm Solution of Non-Monotone Complementarity Problems,” Functional Analysis, Optimization and Mathematical Economics, pp. 217-221, New York: Oxford Univ. Press, 1990.
[14] O.L. Mangasarian, “Arbitrary-Norm Separating Plane,” Operations Research Letters, vol. 24, pp. 15-23, 1999, ftp://ftp.cs.wisc.edu/math-prog/tech-reports 97-07r.ps.
[15] O.L. Mangasarian, “Generalized Support Vector Machines,” Advances in Large Margin Classifiers, A. Smola, P. Bartlett, B. Schölkopf, and D. Schuurmans, eds., pp. 135-146, Cambridge, Mass.: MIT Press, 2000, ftp://ftp.cs.wisc.edu/math-prog/tech-reports 98-14.ps.
[16] O.L. Mangasarian and R.R. Meyer, “Nonlinear Perturbation of Linear Programs,” SIAM J. Control and Optimization, vol. 17, no. 6, pp. 745-752, Nov. 1979.
[17] MATLAB, User's Guide, The MathWorks, Inc., 1994-2001, http:/www.mathworks.com.
[18] T.M. Mitchell, Machine Learning. Boston: McGraw-Hill, 1997.
[19] P.M. Murphy and D.W. Aha, “UCI Machine Learning Repository,” 1992, www.ics.uci.edu/mlearnMLRepository.html.
[20] D.R. Musicant, “NDC: Normally Distributed Clustered Datasets,” 1998, www.cs.wisc.edu/musicant/datandc/.
[21] S. Odewahn, E. Stockwell, R. Pennington, R. Humphreys, and W. Zumach, “Automated Star/Galaxy Discrimination with Neural Networks,” Astronomical J., vol. 103, no. 1, pp. 318-331, 1992.
[22] B.N. Parlett, The Symmetric Eigenvalue Problem. Philadelphia: SIAM, 1998.
[23] B. Schölkopf and A. Smola, Learning with Kernels. Cambridge, Mass.: MIT Press, 2002.
[24] Scilab, free scientific software package, 1990-2004, http:/scilabsoft.inria.fr.
[25] J.A.K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines. Singapore: World Scientific Publishing Co., 2002.
[26] A.N. Tikhonov and V.Y. Arsen, Solutions of Ill-Posed Problems. New York: John Wiley & Sons, 1977.
[27] V.N. Vapnik, The Nature of Statistical Learning Theory, second ed. New York: Springer, 2000.

Index Terms:
Index Terms- Support vector machines, proximal classification, generalized eigenvalues.
Citation:
Olvi L. Mangasarian, Edward W. Wild, "Multisurface Proximal Support Vector Machine Classification via Generalized Eigenvalues," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 69-74, Jan. 2006, doi:10.1109/TPAMI.2006.17
Usage of this product signifies your acceptance of the Terms of Use.