| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
Bayesian Function Learning Using MCMC Methods
December 1998 (vol. 20 no. 12)
pp. 1319-1331
Abstract—The paper deals with the problem of reconstructing a continuous one-dimensional function from discrete noisy samples. The measurements may also be indirect in the sense that the samples may be the output of a linear operator applied to the function (linear inverse problem, deconvolution). In some cases, the linear operator could even contain unknown parameters that are estimated from a second experiment (joint identification-deconvolution problem). Bayesian estimation provides a unified treatment of this class of problems, but the practical calculation of posterior densities leads to analytically intractable integrals. In the paper it is shown that a rigourous Bayesian solution can be efficiently implemented by resorting to a MCMC (Markov chain Monte Carlo) simulation scheme. In particular, it is discussed how the structure of the problem can be exploited in order to improve computational and convergence performances. The effectiveness of the proposed scheme is demonstrated on two classical benchmark problems as well as on the analysis of IVGTT (IntraVenous Glucose Tolerance Test) data, a complex identification-deconvolution problem concerning the estimation of the insulin secretion rate following the administration of an intravenous glucose injection.
[1] 1319 G. Wahba, "Practical Approximate Solutions to Linear Operator Equations When the Data Are Noisy," SIAM J. Numerical Analysis, vol. 14, pp. 651-667, 1977.[2] D. Commenges, "The Deconvolution Problem: Fast Algorithms Including the Preconditioned Conjugate-Gradient to Compute a Map Estimator," Trans. Automatic Control, vol. 29, pp. 229-243, 1984.[3] M. Bertero, "Linear Inverse Problems and Ill-Posed Problems," Advances in Electronics and Electron Physics, vol. 75, pp. 1-120, 1989.[4] G. De Nicolao, G. Sparacino, and C. Cobelli, "Nonparametric Input Estimation in Physiological Systems: Problems, Methods, Case Studies," Automatica, vol. 33, no. 5, pp. 851-869, 1997.[5] J.S. Liu and R. Chen, "Blind Deconvolution via Sequential Imputations," J. Am. Statistical Assoc., vol. 90, pp. 567-576, 1995.[6] R.M. Neal, Bayesian Learning for Neural Networks.New York: Springer-Verlag, 1996.[7] J.V. Beck and K.J. Arnold, Parameter Estimation in Engineering and Science.New York: Wiley, 1977.[8] D.L. Phillips, "A Technique for the Numerical Solution of Certain Integral Equations of the First Kind," J. ACM, vol. 9, pp. 97-101, 1962.[9] S. Twomey, "On the Numerical Solution of Fredholm Integral Equations of the First Kind by the Inversion of the Linear System Produced by Quadrature," J. ACM, vol. 10, pp. 97-101, 1963.[10] A.N. Tychonov and V.Y. Arsenin, Solutions of Ill-Posed Problems.Washington, D.C.: Winston/Wiley, 1977.[11] T. Poggio and F. Girosi, Networks for Approximation and Learning Proc. IEEE, vol. 78, pp. 1481-1497, 1990.[12] P. Craven and G. Wahba, "Smoothing Noisy Data With Spline Functions: Estimating the Correct Degree of Smoothing by the Method of Generalized Cross-Validation," Numerical Mathematics, vol. 31, pp. 377-403, 1979.[13] G. Wahba, Spline Models for Observational Data.Philadelphia: Soc. for Industrial and Applied Mathematics, 1990.[14] D.J.C. MacKay, "Bayesian Interpolation," Neural Computation, vol. 4, pp. 415-447, 1992.[15] C.K.I. Williams and C.E. Rasmussen, "Gaussian Processes for Regression," D.S. Touretsky, M.C. Mozer, and M.E. Hasselmo, eds., Advances in Neural Information Processing Systems.Cambridge, Mass.: MIT Press, 1996.[16] D.G.T. Denison, B.K. Mallick, and A.F.M. Smith, "Automatic Bayesian Curve Fitting," J. Royal Statistical Soc., Series B, vol. 60, pp. 333-350, 1998.[17] C.F. Ansley, R. Kohn, and C.M. Wong, "Nonparametric Spline Regression With Prior Information," Biometrika, vol. 80, pp. 75-88, 1993.[18] W.R. Gilks, S. Richardson, and D.J. Spiegelhalter, Markov Chain Monte Carlo in Practice.London: Chapman&Hall, 1996.[19] P.C. Hansen, "Numerical Tools for Analysis and Solution of Fredholm Integral Equation of the First Kind," Inverse Problems, vol. 8, pp. 849-872, 1992.[20] R.S. Anderssen and P. Bloomfield, "A Time Series Approach to Numerical Differentiation," Technometrics, vol. 16, pp. 69-75, 1974.[21] N. Metropolis, A.W. Rosenbluth, M.N. Rosenbluth, M.N. Teller, and A.H. Teller, "Equations of State Calculations by Fast Computing Machine," J. Chemical Physics, vol. 21, pp. 1,087-1,091, 1953.[22] W.K. Hastings, "Monte Carlo Sampling Methods Using Markov Chain and Their Applications," Biometrika, vol. 57, pp. 97-109, 1970.[23] S. Geman and D. Geman, "Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, pp. 721-741, 1984.[24] N. Shephard and M.K. Pitt, "Likelihood Analysis of Non-Gaussian Measurment Time Series," Biometrika, vol. 84, no. 3, pp. 653-667, 1997.[25] C. Berzuini and C. Larizza, "A Unified Approach for Modeling Longitudinal and Failure Time Data, With Application in Medical Monitoring," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 2, pp. 109-123, Feb. 1996.[26] A.E. Raftery and S.M. Lewis, "Implementing MCMC," W.R. Gilks, S. Richardson, and D.J. Spiegelhalter, eds., Markov Chain Monte Carlo in Practice, chapter 7, pp. 115-130.London: Chapman&Hall, 1996.[27] A. Gelman, "Inference and Monitoring Convergence," W.R. Gilks, S. Richardson, and D.J. Spiegelhalter, eds., Markov Chain Monte Carlo in Practice, chapter 8, pp. 131-144.London: Chapman&Hall, 1996.[28] J. Pearl, Probabilistic Reasoning in Intelligent Systems.Palo Alto, Calif.: Morgan Kauffman, 1988.[29] R. Bellazzi, P. Magni, and G. De Nicolao, "Dynamic Probabilistic Networks for Modelling and Identifying Dynamic Systems: A MCMC Approach," Intelligent Data Analysis—An Int'l J., vol. 1, no. 4, pp.http://www-east.elsevier.comida, 1997.[30] W.R. Gilks and P. Wild, "Adaptive Rejection Sampling for Gibbs Sampling," Applied Statistics, 1992.[31] L. Tierney, "Markov Chains for Posterior Distributions (With Discussion)," Annals Statistics, vol. 22, pp. 1,701-1,762, 1994.[32] B.R. Hunt, "The Inverse Problem of Radiography," Mathematical Biosciences, vol. 8, pp. 161-179, 1970.[33] G. Sparacino and C. Cobelli, "A Stochastic Deconvolution Method to Reconstruct Insulin Secretion Rate After a Glucose Stimulus," IEEE Trans. Biomedical Eng., vol. 43, no. 5, pp. 512-529, 1996.[34] E.T. Shapiro, H. Tillil, A.H. Rubenstein, and K.S. Polonsky, "Peripheral Insulin Parallels Changes in Insulin Secretion More Closely Than C-Peptide After Bolus Intravenous Glucose Administration," J. Clinical Endocrinological Metabolism, vol. 67, pp. 1,094-1,099, 1988.[35] W.J. Gauderman and D.C. Thomas, "Censored Survival Models for Genetic Epidemiology: A Gibbs Sampling Approach," Genetic Epidemiology, vol. 11, pp. 171-188, 1994.[36] C. Litton and C. Buck, "An Archaeological Example: Radiocarbon Dating," W.R. Gilks, S. Richardson, and D.J. Spiegelhalter, eds., Markov Chain Monte Carlo in Practice, chapter 25, pp. 465-480.London: Chapman&Hall, 1996.
Index Terms:
Bayesian estimation, smoothing, inverse problems, system identification, Markov chain Monte Carlo methods, dynamic systems.
Citation:
Paolo Magni, Riccardo Bellazzi, Giuseppe De Nicolao, "Bayesian Function Learning Using MCMC Methods," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1319-1331, Dec. 1998, doi:10.1109/34.735805
