This Article 
 Bibliographic References 
 Add to: 
A Hybrid Neural Network Model for Solving Optimization Problems
February 1993 (vol. 42 no. 2)
pp. 218-227

A hybrid neural network model for solving optimization problems is proposed. An energy function which contains the constraints and cost criteria of an optimization problem is derived, and then the neural network is used to find the global minimum (or maximum) of the energy function, which corresponds to a solution of the optimization problem. The network contains two subnets: a constraint network and a goal network. The constraint network models the constraints of an optimization problem and computes the gradient (updating) value of each neuron such that the energy function monotonically converges to satisfy all constraints of the problem. The goal network points out the direction of convergence for finding an optimal value for the cost criteria. These two subnets ensure that the neural network finds feasible as well as optimal (or near-optimal) solutions. The traveling salesman problem and the Hamiltonian cycle problem are used to demonstrate the method.

[1] S. V. B. Aiyer, M. Niranjan, and F. Fallside, "A theoretical investigation into the performance of the Hopfield model,"IEEE Trans. Neural Networks, vol. 1, pp. 204-215, June 1990.
[2] B. Angeniol, G. De La Croix Vaubois, and J.-Y. Le Texier, "Self-organizing feature maps and the travelling salesman problem,"Neural Networks, vol. I, pp. 289-293, 1988.
[3] G. W. Davis, "Sensitivity analysis in neural net solutions,"IEEE Trans. Syst., Man, Cybern., vol. 19, no. 5, pp. 1078-1082, 1989.
[4] R. Durbin, R. Szeliski, and A. Yuille, "An analysis of the elastic net approach to the traveling salesman problem,"Neural Computat., vol. 1, p. 384, 1989.
[5] H. C. Fu, J. N. Hwang, S. Y. Kung, W. D. Mao, and J. A. Vlontzos, "A universal digital VLSI design for neural networks," inProc. IJCNN'89, Washington DC, June, 1989.
[6] M. R. Garey and D. S. Johnson,Computers and Intractability: A Guide to Theory of NP-Completeness. San Francisco, CA: Freeman, 1979.
[7] K. M. Gutzmann, "Combinatorial optimization using a continuous state Boltzmann machine," inProc. IEEE First Int. Conf. Neural Networks, vol. III, San Diego, CA, June 1987, pp. 721-728.
[8] J. J. Hopfield and D. W. Tank, "Neural composition of decisions optimization problems," vol. 55, pp. 141-152, 1985.
[9] E. Horowitz and S. Sahni,Fundamentals of Computer Algorithms. Rockville, MD: Computer Sci. Press, 1978.
[10] J. Johnson, "A Neural Network Approach to the 3-Satisfiability Problem,"J. Parallel and Distributed Computing, Vol. 6, 1989, pp. 435-439.
[11] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, "Optimization by simulated annealing,"Science, vol. 220, p. 671, 1983.
[12] S. Kirkpatrick and G. Toulouse, "Configuration space analysis of the traveling salesman problem,"J. Phys., vol. 42, p. 1277, 1985.
[13] S.Y. Kung,VLSI Array Processors, Prentice Hall, Englewood Cliffs, N.J. 1988.
[14] B. W. Lee and B. J. Sheu, "Combinatorial optimization using competitive-Hopfield neural network," inProc. Int. Joint Conf. Neural Networks, vol. II, Washington DC, Jan. 1990. pp. 627-630.
[15] R. P. Lippman, "An introduction to computing with neural nets,"IEEE ASSP Msg., vol. 4, pp. 4-22, 1987.
[16] D. G. Luenberger, "Speed of convergence," inLinear and Nonlinear Programming. Reading, MA: Addison-Wesley., 1984, ch. 6, pp. 189-192.
[17] D. G. Luenberger, "Basic descent methods," inLinear and Nonlinear Programming. Reading, MA: Addison-Wesley, 1984, ch. 7, pp. 197-237.
[18] S. Mehta and L. Fulop, "A neural algorithm to solve the Hamiltonian cycle problem," inProc. Int. Joint Conf. Neural Networks, vol. III, San Diego, CA, June 18-22, 1990 pp. 843-849.
[19] H. Muhlenbein, M. Gorges-Schleuter, and O. Kramer, "Evolution algorithms in combinatorial optimization,Parallel Computat., vol. 7, p. 65, 1988.
[20] C. Peterson and B. Soderberg, "A new method for mapping optimization problems onto neural networks,"Int. J. Neural Syst., vol. 1, pp. 3-22, 1989.
[21] C. Peterson, "Parallel distributed approaches to combinatorial optimization: Benchmark studies on the traveling salesman problem,"Neural Computat., M.I.T. Press Journals, vol. 2, pp. 261-269, 1990.
[22] J. Ramanujam and P. Sadydppan, "Parameter identification for constrained optimization using neural networks," inProc. 1988 Connectionist Models Summer School, Morgan Kaufmann, 1988 pp. 154-161.
[23] K. T. Sun and H. C. Fu, "Solving satisfiability problems with neural networks," inProc. IEEE Region 10 Conf. Comput. and Commun. Syst., vol. 1, Hong Kong, Sept. 24-27, 1990, pp. 17-22.
[24] K. T. Sun and H. C. Fu, "A neural network for solving the satisfiability problems," inProc. Int. Comput. Symp. 1990, National Tsing Hua University, Taiwan, R.O.C., Dec. 17-19, 1990 pp. 757-762.
[25] K. T. Sun and H. C. Fu, "An O(n) parallel algorithm for solving the traffic control problem on crossbar switch networks,"Parallel Processing Lett. (PPL), vol. 1, no. 1, pp. 51-58, 1991.
[26] K. T. Sun and H. C. Fu, "A neural network algorithm for solving the traffic control problem in multistage interconnection networks," inProc. Int. Joint Conf. Neural Networks (IJCNN-91), Singapore, Nov. 24-28, 1991 pp. 1136-1141.
[27] K. T. Sun and H. C. Fu, "A neural network approach to restrictive channel routing problems," inProc. Int. Conf. Artificial Networks ICANN'92.
[28] G. A. Taglianeri and E. W. Page, "Solving constraint satisfaction problems with neural networks," inProc. Int. Conf. Neural Networks, San Diego, CA, June 1987, pp. 741-747.
[29] R. J. Williams, "Learning the logic of activation functions," inParallel Distributed Processing, Vol. 1. Cambridge, MA: M.I.T. Press, 1986, ch. 10, pp. 423-443.
[30] G. V. Wilson and G. S. Pawley, "On the stability of the travelling salesman algorithm of Hopfield and Tank,"Biol. Cybern., vol. 58, pp. 63-70, 1988.
[31] X. Xu and W. T. Tsai, "An adaptive neural algorithm for traveling salesman problem," inProc. Int. Joint Conf. Neural Networks, vol. I, Washington, DC, Jan. 1990, pp. 716-719.
[32] A. Yuille, "Generalized deformable models, statistical physics, and matching problems,"Neural Computat., vol. 2, pp. 1-24, 1990.

Index Terms:
hybrid neural network model; optimization problems; energy function; constraints; cost criteria; global minimum; goal network; traveling salesman problem; Hamiltonian cycle problem; constraint handling; neural nets; operations research; optimisation.
K.T. Sun, H.C. Fu, "A Hybrid Neural Network Model for Solving Optimization Problems," IEEE Transactions on Computers, vol. 42, no. 2, pp. 218-227, Feb. 1993, doi:10.1109/12.204794
Usage of this product signifies your acceptance of the Terms of Use.