| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
A Probabilistic Active Support Vector Learning Algorithm
March 2004 (vol. 26 no. 3)
pp. 413-418
Abstract—The paper describes a probabilistic active learning strategy for support vector machine (SVM) design in large data applications. The learning strategy is motivated by the statistical query model. While most existing methods of active SVM learning query for points based on their proximity to the current separating hyperplane, the proposed method queries for a set of points according to a distribution as determined by the current separating hyperplane and a newly defined concept of an adaptive confidence factor. This enables the algorithm to have more robust and efficient learning capabilities. The confidence factor is estimated from local information using the k nearest neighbor principle. The effectiveness of the method is demonstrated on real-life data sets both in terms of generalization performance, query complexity, and training time.
[1] 413 V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.[2] C.K.I. Williams and M. Seeger, Using the Nystrom Method to Speed Up Kernel Machines Proc. Advances in Neural Information Processing System, vol. 14, 2001.[3] M.E. Tipping and A. Faul, Fast Marginal Likelihood Maximization for Sparse Bayesian Models Proc. Int'l Workshop AI and Statistics, 2003.[4] B. Scholkopf, Advances in Kernel Methods Support Vector Learning, C.J.C. Burges and A.J. Smola, eds. MIT Press, 1998.[5] C.J.C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition Data Mining and Knowledge Discovery, vol. 2, no. 2, pp. 1-47, 1998.[6] L. Kaufman, Solving the Quadratic Programming Problem Arising in Support Vector Classification Advances in Kernel Methods Support Vector Learning, B. Scholkopf, C.J.C. Burges, and A. J. Smola, eds. pp. 147-168, MIT Press, 1998.[7] D. Cohn, Z. Ghahramani, and M. Jordan, Active Learning with Statistical Models J. AI Research, vol. 4, pp. 129-145, 1996.[8] D. MacKay, Information Based Objective Function for Active Data Selection Neural Computation, vol. 4, no. 4, pp. 590-604 1992.[9] C. Campbell, N. Cristianini, and A. Smola, Query Learning with Large Margin Classifiers Proc. 17th Int'l Conf. Machine Learning, pp. 111-118, 2000.[10] S. Tong and D. Koller, Support Vector Machine Active Learning with Application to Text Classification J. Machine Learning Research, vol. 2, pp. 45-66, 2001.[11] G. Schohn and D. Cohn, Less Is More: Active Learning with Support Vector Machines Proc. 17th Int'l Conf. Machine Learning, pp. 839-846, 2000.[12] P. Mitra, C.A. Murthy, and S.K. Pal, Data Condensation in Large Data Bases by Incremental Learning with Support Vector Machines Proc. 15th Int'l Conf. Pattern Recognition, pp. 712-715, 2000.[13] M.J. Kearns, Efficient Noise-Tolerant Learning from Statistical Queries Proc. 25th ACM Symp. Theory of Computing, pp. 392-401, 1993.[14] S. Seo, M. Wallat, T. Graepel, and K. Obermayer, Gaussian Process Regression: Active Data Selection and Test Point Rejection Proc. Int'l Joint Conf. Neural Networks, vol. 3, pp. 241-246, 2000.[15] C. Blake and C. Merz, UCI Repository of Machine Learning Databases, http://www.ics.uci.edu/mlearnMLRepository.html , Dept. of Information and Computer Sciences, Univ. of California, Irvine, 1998.[16] J.C. Platt, Fast Training of Support Vector Machines Using Sequential Minimal Optimisation Advances in Kernel Methods Support Vector Learning, B. Scholkopf, C.J.C. Burges, and A.J. Smola, eds. pp. 185-208, MIT Press, 1998.[17] N.A. Sayeed, H. Liu, and K.K. Sung, A Study of Support Vectors on Model Independent Example Selection Proc. First Int'l Conf. Knowledge Discovery and Data Mining, pp. 272-276, 1999.[18] D.P. Mandal, C.A. Murthy, and S.K. Pal, Determining the Shape of a Pattern Class from Sampled Points in$R^2$ Int'l J. General Systems, vol. 20, no. 4, pp. 307-339, 1992.
Index Terms:
Data mining, learning theory, query learning, incremental learning, statistical query model, classification.
Citation:
Pabitra Mitra, C.A. Murthy, Sankar K. Pal, "A Probabilistic Active Support Vector Learning Algorithm," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 3, pp. 413-418, Mar. 2004, doi:10.1109/TPAMI.2004.1262340
