This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Global Optimization for Neural Network Training
March 1996 (vol. 29 no. 3)
pp. 45-54

Many learning algorithms find their roots in function-minimization algorithms that can be classified as local- or global-minimization algorithms. Algorithms that focus on either extreme--local search or global search--do not work well. The authors propose a hybrid method, called NOVEL for Nonlinear Optimization via External Lead, that combines global and local searches to explore the solution space, locate promising regions, and find local minima. To guide exploration of the solution space, it uses a continuous terrain-independent trace that does not get trapped in local minima. NOVEL next uses a locate gradient to attract the search to a local minimum, but the trace pulls it out once little improvement is found. NOVEL then selects one initial point for each promising region and uses these points for a descent algorithm to find local minima. It thus avoids searching unpromising local minima from random starting points using computationally expensive descent algorithms. In an implementation using differential- and difference-equation solvers, NOVEL demonstrated superior performance in five benchmark comparisons against the best global optimization algorithms.

Citation:
Yi Shang, Benjamin W. Wah, "Global Optimization for Neural Network Training," Computer, vol. 29, no. 3, pp. 45-54, March 1996, doi:10.1109/2.485892
Usage of this product signifies your acceptance of the Terms of Use.