This Article 
 Bibliographic References 
 Add to: 
Synchronous and Asynchronous Parallel Simulated Annealing with Multiple Markov Chains
October 1996 (vol. 7 no. 10)
pp. 993-1008

Abstract—Simulated annealing is a general-purpose optimization technique capable of finding an optimal or near-optimal solution in various applications. However, the long execution time required for a good quality solution has been a major drawback in practice. Extensive studies have been carried out to develop parallel algorithms for simulated annealing. Most of them were not very successful, mainly because multiple processing elements (PEs) were required to follow a single Markov chain and, therefore, only a limited parallelism was exploited. In this paper, we propose new parallel simulated annealing algorithms which allow multiple Markov chains to be traced simultaneously by PEs which may communicate with each other. We have considered both synchronous and asynchronous implementations of the algorithms. Their performance has been analyzed in detail and also verified by extensive experimental results. It has been shown that for graph partitioning the proposed parallel simulated annealing schemes can find a solution of equivalent (or even better) quality up to an order of magnitude faster than the conventional parallel schemes. Among the proposed schemes, the one where PEs exchange information dynamically (not with a fixed period) performs best.

[1] S. Kirkpatrick, C.D. Gelatt Jr., and M.P. Vecchi, "Optimization by Simulated Annealing," Science, vol. 220, pp. 671-680, 1983.
[2] U. Faigle and R. Schrader, "On the Convergence of Stationary Distributions in Simulated Annealing Algorithms," Information Processing Letters, vol. 27, pp. 189-194, 1988.
[3] A. Corana et al., "Minimizing Multi-Modal Functions of Continuous Variables with the Simulated Annealing Algorithm," ACM Trans. Mathematical Software, Vol. 13, No. 3, 1987, pp. 262-280.
[4] F. Romeo and Sangiovanni-Vincentelli, "Probabilistic Hill Climbing Algorithms: Properties and Applications," Proc. Chapel Hill Conf. VLSI,Chapel Hill, pp. 393-417, 1985.
[5] R.H.J.M. Otten and L.P.P.P. van Ginneken, "Floorplan Design Using Simulated Annealing," Proc. IEEE Int'l Conf. Computer-Aided Design,Santa Clara, Calif., pp. 96-98, 1984.
[6] B. Hajek, "Cooling Schedules for Optimal Annealing," Mathematics of Operations Research, vol. 13, pp. 311-329, 1988.
[7] E.H.L. Aarts and J.H.M. Korst, Simulated Annealing and Boltzmann Machines. John Wiley&Sons, 1989.
[8] S.A. Kravitz and R.A. Rutenbar, "Placement by Simulated Annealing on a Multi-Processor," IEEE Trans. Computer-Aided Design, vol. 6, pp. 534-549, July 1987.
[9] E.E. Witte, R.D. Chamberlain, and M.A. Franklin, "Parallel Simulated Annealing Using Speculative Computation," IEEE Trans. Parallel and Distributed Systems, vol. 2, pp. 483-494, 1991.
[10] P. Banerjee, M.H. Jones, and J.S. Sargent, “Parallel Simulated Annealing Algorithms for Cell Placement on Hypercube Multiprocessors,” IEEE Trans. Parallel and Distributed Systems, vol. 1, no. 1, Jan. 1990.
[11] P. Roussel-Ragot and G. Dreyfus, "A Problem Independent Parallel Implementation of Simulated Annealing: Model and Experiments," IEEE Trans. Computer-Aided Design, vol. 9, no. 8, pp. 827-835, Aug. 1990.
[12] D.R. Greening, “Parallel Simulated Annealing Techniques,” Physica, vol. 42, pp. 293-306, 1990.
[13] K.S. Natarajan, "Graph-Partitioning on Shared-Memory Multiprocessor Systems," Proc. Int'l Conf. Parallel Processing, vol. 3, pp. 120-124, Aug. 1991.
[14] V.C. Barbosa and E. Gafni, "A Distributed Implementation of Simulated Annealing," J. Parallel and Distributed Computing, vol. 6, pp. 411-434, 1989.
[15] E.H. Aarts and J.H.M. Korst, "Boltzmann Machines as a Model for Parallel Annealing," Algorithmica, vol. 6, pp. 437-465, 1991.
[16] A. Casotto and A.L. Sangiovanni-Vincentelli, "Placement of Standard Cells Using Simulated Annealing on the Connection Machine," Proc. IEEE Int'l Conf. Computer-Aided Design, pp. 350-353,Santa Clara, Calif., 1987.
[17] K.-G. Lee and S.-Y. Lee, "Parallel Simulated Annealing for Finding Minima of Functions on a Hypercube Multiprocessor," SIAM Ann. Meeting,Chicago, July 1990.
[18] S.-Y. Lee, H.D. Chiang, K. -G. Lee, and B.Y. Ku, "Parallel Power System Transient Stability Analysis on Hypercube Multiprocessors," IEEE Trans. Power Systems, vol. 6, no. 3, pp. 1,337-1,343, Aug. 1991.
[19] M.D. Huang, F. Romeo, and A. Sangiovanni-Vincentelli, "An Efficient General Cooling Schedule for Simulated Annealing," Proc. Int'l Conf. Computer-Aided Design, pp. 381-384, Nov. 1986.
[20] D.E. Goldberg, Genetic Algorithms. Addison-Wesley, 1989.
[21] M.R. Irving and M.J.H. Sterling, "Optimal Network Tearing Using Simulated Annealing," IEE Proc., vol. 137, no. 1, pp. 69-72, Jan. 1990.
[22] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness.New York: W.H. Freeman, 1979.
[23] K.-G. Lee and S.-Y. Lee, "Efficient Parallelization of Simulated Annealing Using Multiple Markov Chains: An Application to Graph Partitioning," Proc. Int'l Conf. Parallel Processing,St. Charles, Ill., Aug. 1992.
[24] S.-Y. Lee and K.-G. Lee, "Asynchronous Communication of Multiple Markov Chains in Parallel Simulated Annealing," Proc. Int'l Conf. Parallel Processing,St. Charles, Ill., Aug. 1992.

Index Terms:
Asynchronous communication, graph partitioning, multiple Markov chains, parallel algorithm, simulated annealing, solution quality, speed-up.
Soo-Young Lee, Kyung Geun Lee, "Synchronous and Asynchronous Parallel Simulated Annealing with Multiple Markov Chains," IEEE Transactions on Parallel and Distributed Systems, vol. 7, no. 10, pp. 993-1008, Oct. 1996, doi:10.1109/71.539732
Usage of this product signifies your acceptance of the Terms of Use.