| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
Cardinal Interpolation
September 2007 (vol. 29 no. 9)
pp. 1538-1545
A Bayesian probability density for an interpolating function is developed, and its desirable properties and practical potential are demonstrated. This density has an often needed but previously unachieved property, here called cardinal interpolation, which ensures extrapolation to the density of the least squares linear model. In particular, the mean of the cardinal interpolation density is a smooth function that intersects given (x, y) points and which extrapolates to their least squares line, and the variance of this density is a smooth function that is zero at the point x values, that increases with distance from the nearest point x value, and that extrapolates to the well-known quadratic variance function for the least squares line. The new cardinal interpolation density is developed for Gaussian radial basis interpolators using fully Bayesian methods that optimize interpolator smoothness. This optimization determines the basis function widths and yields an interpolating density that is non-Gaussian except for large magnitude x and which is therefore not the outcome of a Gaussian process. Further new development shows that the salient property of extrapolation to the density of the least squares linear model can be achieved for more general approximating (not just interpolating) functions.
[1] O. Bousquet and A. Elisseeff, “Stability and Generalization,” J.Machine Learning Research, vol. 2, pp. 499-526, 2002.
[2] T. Poggio, R. Rifkin, S. Mukherjee, and P. Niyogi, “General Conditions for Predictivity in Learning Theory,” Nature, vol. 428, pp. 419-422, 2004.
[3] C. Rasmussen and C.K.I. Williams, Gaussian Processes for Machine Learning. MIT Press, 2006.
[4] M.L. Stein, Interpolation of Spatial Data: Some Theory for Kriging. Springer, 1999.
[5] D.J.C. MacKay, “Bayesian Interpolation,” Neural Computation, vol. 4, pp. 415-447, 1992.
[6] C. Bishop, Neural Networks for Pattern Recognition. Oxford, 1995.
[7] C. Rasmussen, “The Gaussian Processes Web Site,” http:/www.gaussianprocess.org, 2006.
[8] D.J.C. MacKay, Information Theory, Inference, and Learning Algorithms. Cambridge, 2003.
[9] M. Seeger, “Gaussian Processes for Machine Learning,” Int'l J. Neural Systems, vol. 14, pp. 69-106, 2004.
[10] N. Christianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge, 2000.
[11] J. Quinonero-Candela and C. Rasmussen, “A Unifying View of Sparse Approximate Gaussian Process Regression,” J. Machine Learning Research, vol. 6, pp. 1939-1959, 2005.
[12] M. Tipping, “Sparse Bayesian Learning and the Relevance Vector Machine,” J. Machine Learning Research, vol. 1, pp. 211-244, 2001.
[13] M. Benjamin, R. Rigby, and D. Stasinopoulos, “Generalized Autoregressive Moving Average Models,” J. Am. Statistical Assoc., vol. 98, pp. 214-233, 2003.
[14] I. DiMatteo, C. Genovese, and R. Kass, “Bayesian Curve Fitting with Free Knot Splines,” Biometrika, vol. 88, pp. 1055-1071, 2002.
[15] C. Bailer-Jones, H. Bhadeshia, and D.J.C. MacKay, “Gaussian Process Modelling of Austenite Formation in Steel,” Materials Science and Technology, vol. 15, pp. 287-294, 1999.
[16] F. Aires, C. Prigent, and W. Rossow, “Neural Network Uncertainty Assessment Using Bayesian Statistics: A Remote Sensing Application,” Neural Computation, vol. 16, pp. 2415-2458, 2004.
[17] I. Rivals and L. Personnaz, “Construction of Confidence Intervals for Neural Networks Based on Least Squares Estimation,” Neural Networks, vol. 13, pp. 463-484, 2000.
[18] M. Sugiyama, M. Kawanabe, and K. Miller, “Trading Variance Reduction with Unbiasedness: The Regularized Subspace Information Criterion for Robust Model Selection in Kernel Regression,” Neural Computation, vol. 16, pp. 1077-1104, 2004.
[19] D. Sivia, Data Analysis: A Bayesian Tutorial. Oxford, 1996.
[20] D. Wackerly, W. Mendenhall, and R. Scheaffer, Mathematical Statistics with Applications, sixth ed. Duxbury, 2001.
[21] C. Andrieu, N. deFreitas, and A. Doucet, “Robust Full Bayesian Learning for Radial Basis Networks,” Neural Computation, vol. 13, pp. 2359-2407, 2001.
[22] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. Springer, 2001.
[23] H.K.H. Lee, Bayesian Nonparametrics via Neural Networks. SIAM, 2004.
[24] B. Scholkopf and A. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, 2002.
[25] T. Zhang, “Learning Bounds for Kernel Regression Using Effective Data Dimensionality,” Neural Computation, vol. 17, pp. 2077-2098, 2005.
[26] A. Weigend and N. Gershenfeld, Time Series Prediction: Forecasting the Future and Understanding the Past. Addison-Wesley, 1994.
[27] W. Rudin, Principles of Mathematical Analysis, third ed. McGraw, 1976.
Index Terms:
Bayesian statistics, Interpolation, Modeling and prediction, Probability and statistics, Regression
Citation:
Steven C. Gustafson, David R. Parker, Richard K. Martin, "Cardinal Interpolation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1538-1545, June 2007, doi:10.1109/TPAMI.2007.1170
