This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Fitting Optimal Piecewise Linear Functions Using Genetic Algorithms
July 2000 (vol. 22 no. 7)
pp. 701-718

Abstract—Constructing a model for data in ${\cal R}^{2}$ is a common problem in many scientific fields, including pattern recognition, computer vision, and applied mathematics. Often little is known about the process which generated the data or its statistical properties. For example, in fitting a piecewise linear model, the number of pieces, as well as the knot locations, may be unknown. Hence, the method used to build the statistical model should have few assumptions, yet, still provide a model that is optimal in some sense. Such methods can be designed through the use of genetic algorithms. In this paper, we examine the use of genetic algorithms to fit piecewise linear functions to data in ${\cal R}^{2}$. The number of pieces, the location of the knots, and the underlying distribution of the data are assumed to be unknown. We discuss existing methods which attempt to solve this problem and introduce a new method which employs genetic algorithms to optimize the number and location of the pieces. Experimental results are presented which demonstrate the performance of our method and compare it to the performance of several existing methods. We conclude that our method represents a valuable tool for fitting both robust and nonrobust piecewise linear functions.

[1] P.A. Kokkonis and V. Leute, “Least Squares Splines Approximation Applied to Multicomponent Diffusion Data,” Computational Materials Science, vol. 6, no. 1, pp. 103-111, 1996.
[2] C.L. Karr and B. Weck, “Improved Computer Modelling Using Least Median Squares Curve Fitting and Genetic Algorithms,” Fluid/Practice Separation J., vol. 8, no. 2, pp. 117-124, 1995.
[3] R.L. Eubank, Spline Smoothing and Nonparametric Regression. New York: Marcel Dekker, 1988.
[4] B. Cheng and D.M. Titterington, “Neural Networks: A Review from a Statistical Perspective,” Statistical Science, vol. 9, pp. 2-30, 1994.
[5] K. Warwick and R. Craddock, “An Introduction to Radial Basis Functions for System Identification a Comparison with Other Neural Network Methods,” Proc. IEEE Conf. Decision and Control, vol. 1, pp. 464-469 1996.
[6] C.L. Karr, D.A. Stanley, and B.J. Scheiner, “Genetic Algorithm Applied to Least-Squares Curve Fitting,” Report of Investigations 9339, U.S. Dept. of the Interior, Bureau of Mines, 1991.
[7] S. Chatterjee, M. Laudato, and L.A. Lynch, “Genetic Algorithms and Their Statistical Applications: An Introduction,” Computational Statistics and Data Analysis, vol. 22, no. 6, pp. 633-651, 1996.
[8] C.A. Murthy and J. Pittman, “Optimal Line Fitting Using Genetic Algorithms,” Technical Report 99-02, Dept. of Statistics, The Pennsylvania State Univ., University Park, 1997.
[9] Handbook of Genetic Algorithms, L. Davis ed. New York: Van Nostrand Reinhold, 1991.
[10] D. Chen and R. Jain, “A Robust Back Propagation Learning Algorithm for Function Approximation,” IEEE Trans. Neural Networks, vol. 5, pp. 467-479, 1994.
[11] B.D. Ripley, Pattern Recognition and Neural Networks. New York: Cambridge Univ. Press, 1996.
[12] R.L. Eubank and P. Speckman, “Curve Fitting by Polynomial-Trigonometric Regression,” Biometrika, vol. 77, no. 1, pp. 1-9, 1990.
[13] P. Craven and G. Wahba, “Smoothing Noisy Data with Spline Functions,” Numerische Mathematik, vol. 31, no. 4, pp. 377-403, 1979.
[14] H. Schwetlick and T. Schütze, “Least Squares Approximation by Splines with Free Knots,” BIT, vol. 35, pp. 361-384, 1995.
[15] C. de Boor and J. Rice, “Least Squares Cubic Spline Approximation II—Variable Knots,” CSD Technical Report 21, Dept. of Computer Science, Purdue Univ., West Lafayette, Ind., 1968.
[16] P. Dierckx, Curve and Surface Fitting with Splines. Clarendon Press, 1993.
[17] L. Breiman, “Neural Networks: A Review from a Statistical Perspective,” Statistical Science, vol. 9, p. 38, 1994.
[18] D.L. Barrow, C.K. Chui, P.W. Smith, and J.D. Ward, “Unicity of Best Mean Approximation by Second Order Splines with Variable Knots,” Math. Computing, vol. 32, pp. 1,131-1,143, 1978.
[19] M.J. Baines, “Algorithms for Optimal Discontinuous Piecewise Linear and Constant${\cal{L}}_{2}$Fits to Continuous Functions with Adjustable Nodes in One and Two Dimensions,” Math. Computing, vol. 62, no. 206, pp. 645-669, 1994.
[20] Y. Tourigny and M.J. Baines, “Analysis of an Algorithm for Generating Locally Optimal Meshes for${\cal{L}}_{2}$Approximation by Discontinuous Piecewise Polynomials,” Math. Computing, vol. 66, no. 218, pp. 623-650, 1997.
[21] P.D. Loach and A.J. Wathen, “On the Best Least Squares Approximation of Continuous Functions Using Linear Splines with Free Knots,” IMA J. Numerical Analysis, vol. 11, no. 3, pp. 393-409, 1991.
[22] C.K. Chui, P.W. Smith, and J.D. Ward, “On the Smoothness of Best${\cal{L}}_{2}$Approximants from Nonlinear Spline Manifolds,” Math. Computing, vol. 31, pp. 17-23, 1977.
[23] D.G.T. Denison, B.K. Mallick, and A.F.M. Smith, “Automatic Bayesian Curve Fitting,” J. of the Royal Statistical Soc. Series B, vol. 60, no. 2, pp. 333-350, 1998.
[24] D. Bhandari, C.A. Murthy, and S.K. Pal, “Genetic Algorithm with Elitist Model and Its Convergence,” Int'l J. Pattern Recognition and Artifical Intelligence, vol. 10, no. 6, pp. 731-747, 1996.
[25] P. Vankeerberghen et al., “Robust Regression and Outlier Detection for Non-Linear Models Using Genetic Algorithms,” Chemometrics and Intelligent Laboratory Systems, vol. 28, no. 1, pp. 73-87, 1995.
[26] S. Bandyopadhyay, C.A. Murthy, and S.K. Pal, “Pattern Classification Using Genetic Algorithms: Determination of${\cal H}$,” Pattern Recognition Letters, vol. 19, pp. 1,171-1,181, 1998.
[27] T. Bäck, “Optimal Mutation Rates in Genetic Search,” Proc Fifth Int'l Conf. Genetic Algorithms, 1993.
[28] K.A. DeJong, “An Analysis of the Behavior of a Class of Genetic Adaptive Systems,” doctoral dissertation, Dept. of Electrical Eng. and Computer Science, Univ. of Michigan, Ann Arbor, 1975.
[29] P.J. Huber, Robust Statistics. New York: Wiley, 1981.
[30] S. Bandyopadhyay, C.A. Murthy, and S.K. Pal, “Pattern Classification with Genetic Algorithms,” Pattern Recognition Letters, vol. 16, pp. 801-808, 1995.
[31] C. de Boor, A Practical Guide to Splines. New York: Springer-Verlag, 1978.
[32] R.A. Becker, “The New S Language,” Bell Telephone Laboratories, Murray Hill, N.J., 1988.
[33] Visual Numerics, Inc., IMSL Fortran Numerical Libraries version 10. 1997.
[34] S. Weisberg, Applied Linear Regression. New York: Wiley, 1985.
[35] R. Ash, Real Analysis and Probability. New York: Academic Press, 1972.
[36] J. Pittman, “Adaptive Splines and Genetic Algorithms,” J. of Computational and Graphical Statistics, to appear.

Index Terms:
Genetic algorithms, optimization, statistical data analysis, splines.
Citation:
Jennifer Pittman, C.A. Murthy, "Fitting Optimal Piecewise Linear Functions Using Genetic Algorithms," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 7, pp. 701-718, July 2000, doi:10.1109/34.865188
Usage of this product signifies your acceptance of the Terms of Use.