This Article 
 Bibliographic References 
 Add to: 
Incremental Learning With Sample Queries
August 1998 (vol. 20 no. 8)
pp. 883-888

Abstract—The classical theory of pattern recognition assumes labeled examples appear according to unknown underlying class conditional probability distributions where the pattern classes are picked randomly in a passive manner according to their a priori probabilities. This paper presents experimental results for an incremental nearest-neighbor learning algorithm which actively selects samples from different pattern classes according to a querying rule as opposed to the a priori probabilities. The amount of improvement of this query-based approach over the passive batch approach depends on the complexity of the Bayes rule.

[1] D. Angluin, "Queries and Concept Learning," Machine Learning, vol. 2, pp. 319-342, 1988.
[2] B.W. Silverman, M.C. Jones, E. Fix, and J.l. Hodges, "An Important Contribution to Nonparametric Discriminant Analysis and Density Estimation—Commentary on Fix and Hodges" (1951). International Statistical Review, vol. 57, no. 3, pp. 233-247, 1989.
[3] D. Cohn, "Neural Network Exploitation Using Optimal Experiment Design," Neural Networks, vol. 9, no. 6, pp. 1,071-1,083, 1996.
[4] D. Cohn, L. Atlas, and R. Ladner, "Improving Generalization With Active Learning," Machine Learning, vol 15, pp. 201-221, 1994.
[5] L. Devroye, L. Gyorfi, and G. Lugosi, "A Probabilistic Theory of Pattern Recognition," Springer Verlag, 1996.
[6] K. Fukuda, "Frequently Asked Questions in Geometric Computation," Technical Report, Swiss Federal Inst. of Tech nology, Lausanne, 1997. Available at .
[7] K. Fukuda cdd+ Reference Manual. Inst. for Operations Research, Swiss Federal Inst. of Tech nology, Zurich, Switzerland, 1995. Available at
[8] G.W. Gates, “The Reduced Nearest Neighbor Rule,” IEEE Trans. Information Theory, vol. 18, no. 3, pp. 431-433 1972.
[9] P.E. Hart, “The Condensed Nearest Neighbor Rule,” IEEE Trans. Information Theory, vol. 14, no. 3, pp. 515-516, 1968.
[10] J. O'Rourke, Computational Geometry in C.Cambridge, Mass.: University Press, 1994.
[11] F.P. Preparata, M.I. Shamos., Computational Geometry—An Introduction.New York: Springer-Verlag, 1985.
[12] J. Ratsaby, "Learning Classification with Sample Queries," Submitted to Information and Computation, 1997. Available at
[13] R.L. Rivest, B. Eisenberg, "On the Sample Complexity of Pac-Learning Using Random and Chosen Examples," Proc. 1990 Workshop on Computational Learning Theory, pp. 154-162,San Maeto, CA, 1990.
[14] V.N. Vapnik, Estimation of Dependences Based on Empirical Data.Berlin: Springer-Verlag, 1982.

Index Terms:
Incremental learning, sample querying, nearest-neighbor algorithm, active learning, model selection.
Joel Ratsaby, "Incremental Learning With Sample Queries," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 8, pp. 883-888, Aug. 1998, doi:10.1109/34.709619
Usage of this product signifies your acceptance of the Terms of Use.