This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Convergence and Application of Online Active Sampling Using Orthogonal Pillar Vectors
September 2004 (vol. 26 no. 9)
pp. 1197-1207
Jong-Min Park, IEEE Computer Society
The analysis of convergence and its application is shown for the Active Sampling-at-the-Boundary method applied to multidimensional space using orthogonal pillar vectors. Active learning method facilitates identifying an optimal decision boundary for pattern classification in machine learning. The result of this method is compared with the standard active learning method that uses random sampling on the decision boundary hyperplane. The comparison is done through simulation and application to the real-world data from the UCI benchmark data set. The boundary is modeled as a nonseparable linear decision hyperplane in multidimensional space with a stochastic oracle.

[1] V.V. Federov, Theory of Optimal Experiments. Academic Press, 1972.
[2] S. Tong and D. Koller, Support Vector Machine Active Learning with Applications to Text Classification Proc. 17th Int'l Conf. Machine Learning, 2000.
[3] C. Campbell, N. Cristianini, and A. Smola, Query Learning with Large Margin Classifiers Proc. 17th Int'l Conf. Machine Learning, 2000.
[4] M. Warmuth, G. Rätsch, M. Mathieson, J. Liao, and C. Lemmen, Active Learning in the Drug Discovery Process Advances in Neural Information Processing Systems, T. Dietterich, S. Becker, and Z. Ghahramani, eds., vol. 14, 2002.
[5] G. Schohn and D. Cohn, Less Is More: Active Learning with Support Vector Machines Proc. 17th Int'l Conf. Machine Learning, pp. 839-846, 2000.
[6] D. Cohn, L. Atlas, and R. Ladner, Training Connectionist Networks with Queries and Selective Sampling Advances in Neural Information Processing Systems, vol. 2, 1990.
[7] D. Cohn, Z. Ghahramani, and M. Jordon, Active Learning with Statistical Models Advances in Neural Information Processing Systems, vol. 7, pp. 705-712, 1995.
[8] K. Fukumizu, Active Learning in Multilayer Percetrons Advances in Neural Information Processing Systems, vol. 8, pp. 295-302, 1996.
[9] D.A. Cohn, Neural Network Exploration Using Optimal Experiment Design Advances in Neural Information Processing Systems, vol. 6, 1994, ftp://archive.cis.ohio-state.edu/pub/neuroprose cohn.explore.ps.Z.
[10] J.-N. Hwang, J.J. Choi, S. Oh, and R.J.M. II, Query-Based Learning Applied to Partially Trained Multilayer Perceptrons IEEE Trans. Neural Networks, vol. 2, no. 1, pp. 131-136, Jan. 1991.
[11] Y. Kabashima and S. Shinomoto, Incremental Learning with and Without Queries in Binary Choice Problems Proc. Int'l Joint Conf. Neural Networks, vol. 2, pp. 1637-1640, 1993.
[12] D. MacKay, Information-Based Objective Functions for Active Data Selection Neural Computation, vol. 4, no. 4, pp. 590-604, 1992.
[13] M. Plutowski and H. White, Active Selection of Training Examples for Network Learning in Noiseless Environments Technical Report CS91-180, Univ. of California-San Diego, Feb. 1990.
[14] W. Kinzel and P. Rujan, Improving a Network Generalization Ability by Selecting Examples Europhysics Letters, vol. 13, no. 5, pp. 473-477, 1990.
[15] T. Watkin and A. Rau, Selecting Examples for Perceptrons J. Physics A: Mathematical and General, vol. 25, pp. 113-121, 1992.
[16] P. Sollich, Learning from Minimum Entropy Queries in a Large Committee Machine Physical Rev. E, vol. 53, pp. R2060-R2063, 1996, .
[17] J.-M. Park and Y.-H. Hu, On-Line Learning for Active Pattern Recognition IEEE Signal Processing Letters, Nov. 1996,
[18] C.J.C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition Data Mining and Knowledge Discovery, U. Fayyad, ed., Kluwer Academic Publishers, 1998, ftp://archive.cis.ohio-state.edu/pub/neuroprose/ sollich. queries_comm_machine.ps.Zhttp:/ /attila.sdsu.edu/jong-min/publications/ spletter_all.ps.http://svm.research. bell-labs.com/ paperstutorial_web_page.ps.gz .
[19] S. Karlin and H.M. Taylor, A First Course in Stochastic Processes, second ed. Academic Press, Inc., 1975.
[20] P. Murphy and D. Aha, UCI Repository of Machine Learning Databases technical report, Dept. of Information and Computer Science, Univ. of California, Irvine, 1994.
[21] H. Wang, D. Bell, and F. Murtagh, Axiomatic Approach to Feature Subset Selection Based on Relevance IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 3, pp. 271-277, Mar. 1999.
[22] I. Jolliffe, Principal Component Analysis. Springer-Verlag, 1986.
[23] J.-M. Park and H.-D. Yae, Analysis of Active Feature Selection in Optic Nerve Data Using Labeled Fuzzy C-Means Clustering IEEE Int'l Conf. Fuzzy Systems, 2002, http://attila.sdsu.edu/jong-min/publications svmdm_wcci2002.ps.
[24] J.-M. Park, J. Reed, and Q. Zhou, Active Feature Selection in Optic Nerve Data Using Support Vector Machines Int'l Joint Conf. Neural Networks, 2002.

Index Terms:
Active learning, machine learning, pattern classification.
Citation:
Jong-Min Park, "Convergence and Application of Online Active Sampling Using Orthogonal Pillar Vectors," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1197-1207, Sept. 2004, doi:10.1109/TPAMI.2004.61
Usage of this product signifies your acceptance of the Terms of Use.