Issue No. 05 - May (2006 vol. 28)
Richard A. Olshen , IEEE
Nonparametric neighborhood methods for learning entail estimation of class conditional probabilities based on relative frequencies of samples that are "near-neighbors” of a test point. We propose and explore the behavior of a learning algorithm that uses linear interpolation and the principle of maximum entropy (LIME). We consider some theoretical properties of the LIME algorithm: LIME weights have exponential form; the estimates are consistent; and the estimates are robust to additive noise. In relation to bias reduction, we show that near-neighbors contain a test point in their convex hull asymptotically. The common linear interpolation solution used for regression on grids or look-up-tables is shown to solve a related maximum entropy problem. LIME simulation results support use of the method, and performance on a pipeline integrity classification problem demonstrates that the proposed algorithm has practical value.
Nonparametric statistics, probabilistic algorithms, pattern recognition, maximum entropy, linear interpolation.
Maya R. Gupta, Robert M. Gray, Richard A. Olshen, "Nonparametric Supervised Learning by Linear Interpolation with Maximum Entropy", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 28, no. , pp. 766-781, May 2006, doi:10.1109/TPAMI.2006.101