This Article 
 Bibliographic References 
 Add to: 
State of the Art in Parallel Search Techniques for Discrete Optimization Problems
January/February 1999 (vol. 11 no. 1)
pp. 28-35

Abstract—Discrete optimization problems arise in a variety of domains, such as VLSI design, transportation, scheduling and management, and design optimization. Very often, these problems are solved using state space search techniques. Due to the high computational requirements and inherent parallel nature of search techniques, there has been a great deal of interest in the development of parallel search methods since the dawn of parallel computing. Significant advances have been made in the use of powerful heuristics and parallel processing to solve large-scale discrete optimization problems. Problem instances that were considered computationally intractable only a few years ago are routinely solved currently on server-class symmetric multi-processors and small workstation clusters. Parallel game-playing programs are challenging the best human minds at games like chess. In this paper, we describe the state of the art in parallel algorithms used for solving discrete optimization problems. We address heuristic and nonheuristic techniques for searching graphs as well as trees, and speedup anomalies in parallel search that are caused by the inherent speculative nature of search techniques.

[1] S. Arvindam, V. Kumar, and V. Nageshwara Rao, "Floorplan Optimization on Multiprocessors," Proc. 1989 Int'l Conf. Computer Design, 1989.
[2] R.E. Bixby, W. Cook, A. Cox, and E. Lee, "Parallel Mixed Integer Programming," Technical Report CRPC TR 95554, Center for Research on Parallel Computation, research mo nograph, 1995.
[3] A. Brungger, A. Marzetta, J. Clausen, and M. Perregaard, "Solving Large-Scale QAP Problems in Parallel with the Search Library Zram," J. Parallel and Distributed Computing, vol. 50, pp. 157-169, 1998.
[4] B.L. Cun and C. Roucairol, "BOB: A Unified Platform for Implementing Branch-and-Bound Like Algorithms," Technical Report N95/16, Universiti de Versailles Saint Quentin, 1995; also available at< crbob_us.html>.
[5] E.W. Dijkstra, W.H. Seijen, and A.J.M. Van Gasteren, "Derivation of a Termination Detection Algorithm for a Distributed Computation," Information Processing Letters, vol. 16, no. 5, pp. 217-219, 1983.
[6] S. Dutt and N.R. Mahapatra, Scalable Load-Balancing Strategies for Parallel a* Algorithms," J. Parallel and Distributed Computing, vol. 22, no. 3, pp. 488-505, special issue on scalability of parallel algorithms and architectures, Sept. 1994.
[7] J. Eckstein, "Parallel Branch-and-Bound Methods for Mixed-Integer Programming on the cm-5," SIAM J. Optimization, vol. 4, no. 4, pp. 794-814, 1994.
[8] J. Eckstein, "Distributed Versus Centralized Storage and Control for Parallel Branch and Bound: Mixed Integer Programming on the cm-5," Computational Optimization and Applications, vol. 7, no. 2, pp. 199-220, 1997.
[9] M. Evett, J. Hendler, A. Mahanti, and D. Nau, "PRA*: A Memory-Limited Heuristic Search Procedure for the Connection Machine," Proc. Third Symp. Frontiers of Massively Parallel Computation, pp. 145-149, 1990.
[10] C. Ferguson and R. Korf, "Distributed Tree Search and its Application to Alpha-Beta Pruning," Proc. 1988 Nat'l Conf. Artificial Intelligence, Aug. 1988.
[11] Solving Combinatorial Optimization Problems in Parallel: Methods and Techniques, A. Ferreira and P. Pardalos, eds., Lecture Notes in Computer Science No. 1,054, State-of-the-Art Surveys, Springer-Verlag, 1996.
[12] R. Finkel and U. Manber, "Dib—A distributed implementation of backtracking," ACM Trans. Programming Languages and Systems, vol. 9, pp. 235-255, Apr. 1987.
[13] M. Furuichi, K. Taki, and N. Ichiyoshi, "A Multi-Level Load-Balancing Scheme for OR-Parallel Exhaustive Search Programs on the Multi-PSI," Proc. Second ACM SIGPLAN Symp. Principles and Practice of Parallel Programming, pp. 50-59, 1990.
[14] S. Hamilton and L. Garber, "Deep Blue's Hardware-Software Synergy," Computer, vol. 30, no. 10, pp. 29-35, Oct. 1997.
[15] F.-H. Hsu, "Large-Scale Parallelization of Alpha-Beta Search: An Algorithmic and Architectural Study with Computer Chess," PhD thesis, technical report, Carnegie Mellon Univ., Pittsburgh, Pa., 1990.
[16] F.-H. Hsu, M.S. Campbell, and A.J. Hoane, "Deep Blue System Overview," Proc. Int'l Conf. Supercomputing,Barcelona, Spain, pp. 240-244, 1995.
[17] V.K. Janakiram, D.P. Agrawal, and R. Mehrotra, "A Randomized Parallel Backtracking Algorithm," IEEE Trans. Computers, vol. 37, no. 12, 1988.
[18] L. Johnson, G. Nemhauser, and M. Savelsbergh, "Progress in Integer Programming: An Exposition," technical report, School of Industrial and Systems Engineering, Georgia Inst. of Tech nology, 1997; also available from< mwps wps.html >.
[19] L.V. Kale and V. Saletore, "Parallel State-Space Search for a First Solution with Consistent Linear Speedups," Int'l J. Parallel Programming, vol. 3, Aug. 1990.
[20] L.N. Kanal and V. Kumar, Search in Artificial Intelligence.New York: Springer-Verlag, 1988.
[21] R. Karp and Y. Zhang, "Randomized Parallel Algorithms for Backtrack Search and Branch-and-Bound Computation," J. ACM, vol. 40, pp. 765-789, 1993.
[22] G. Karypis and V. Kumar, "Unstructured Tree Search on SIMD Parallel Computers," IEEE Trans. Parallel and Distributed Systems, vol. 5, pp. 1,057-1,072, 1994.
[23] V. Kumar and L.N. Kanal, "A General Branch-and-Bound Formulations for Understanding and Synthesizing and/or Tree Search Procedures," Artificial Intelligence, vol. 21, pp. 179-198, 1983.
[24] V. Kumar, A. Grama, A. Gupta, and G. Karypis, Introduction to Parallel Computing: Algorithm Design and Analysis.Redwood City, Calif.: Benjamin/Cummings, Addison-Wesley, 1994.
[25] V. Kumar, A. Grama, and V. Nageshwara Rao, "Scalable Load-Balancing Techniques for Parallel Computers," J. Parallel and Distributed Computing, vol. 22, no. 1, pp. 60-79, July 1994.
[26] T.H. Lai and S. Sahni, "Anomalies in Parallel Branch and Bound Algorithms," Comm. ACM, pp. 594-602, 1984.
[27] E.K. Lee and J.E. Mitchell, "Computational Experience of an Interior-Point Algorithm in a Parallel Branch-and-Cut Framework," Proc. SIAM Conf. Parallel Processing for Scientific Computing, 1997.
[28] G.J. Li and B.W. Wah, "Computational Efficiency of Combinatorial Or-Tree Searches," IEEE Trans. Software Eng., vol. 16, no. 1, pp. 13-31, Jan. 1990.
[29] G.-J. Li and B.W. Wah, "Coping with Anomalies in Parallel Branch-and-Bound Algorithms," IEEE Trans. Computers, vol. 35, June 1986.
[30] N.R. Mahapatra and S. Dutt, "Scalable Global and Local Hashing Strategies for Duplicate Pruning in Parallel a* Graph Search," IEEE Trans. Parallel and Distributed Systems, vol. 8, no. 7, July 1997.
[31] B. Mans, T. Mautor, and C. Roucairol, "A Parallel Depth First Search Branch and Bound for the Quadratic Assignment Problem," European J. Operational Research, vol. 81, no. 3, pp. 617-628, 1995.
[32] B. Mans and C. Roucairol, "Performances of Parallel Branch and Bound Algorithms with Best-First Search," Discrete Applied Math., vol. 66, no. 1, pp. 57-76, Apr. 1996.
[33] G. Manzini and M. Somalvico, "Probabilistic Performance Analysis of Heuristic Search Using Parallel Hash Tables," Proc. Int'l Symp. Artificial Intelligence and Math., 1990.
[34] T.A. Marsland and M. Campbell, "Parallel Search of Strongly Ordered Game Trees," Computing Surveys, vol. 14, pp. 533-551, 1982.
[35] D.L. Miller and J.F. Pekny, "The Role of Performance Metrics for Parallel Mathematical Programming Algorithms," ORSA J. Computing, vol. 5, no. 1, 1993.
[36] M. Padberg and G. Rinaldi, "A Branch-and-Cut Algorithm for the Resolution of Large-Scale Symmetric Traveling Salesman Problems," SIAM Rev., vol. 33, pp. 60-100, 1991.
[37] P. Pardalos, Y. Li, K.G. Ramakrishna, and M.G.C. Resende, "Lower Bounds for the Quadratic Assignment Problem," Annals Operations Research, vol. 50, pp. 387-411, special volume on applications of combinatorial optimization, 1994.
[38] J. Pearl, Heuristics: Intelligent Search Strategies for Computer Problem Solving, Addison-Wesley, Reading, Mass., 1984.
[39] I. Pramanick and J.G. Kuhl, "An Inherently Parallel Method for Heuristic Problem-Solving: Part I—General Framework," IEEE Trans. Parallel and Distributed Systems, vol. 6, no. 10, Oct. 1995.
[40] B. Monien, R. Feldmann, and P. Mysliwietz, "Studying Overheads in Massively Parallel Min/Max-Tree Evaluation," Proc. Sixth ACM Symp. Parallel Algorithms and Architectures, pp. 94-103, 1994.
[41] A.G. Ranade, "Optimal Speedup for Backtrack Search on a Butterfly Network," Proc. Third ACM Symp. Parallel Algorithms and Architectures, 1991.
[42] V. Rao and V. Kumar,“Concurrent access of priority queues,” IEEE Trans. Computers, vol. 37, no. 12, pp. 1,657-1,665, 1988.
[43] V. Nageshwara Rao and V. Kumar, "On the Efficiency of Parallel Backtracking," IEEE Trans. Parallel and Distributed Systems, vol. 4, no. 4, pp. 427-437, Apr. 1993; also available as Technical Report TR 90-55, Dept. of Computer Science, Univ. of Minnesota, Minneapolis.
[44] C. Roucairol, "A Parallel Branch and Bound Algorithm for the Quadratic Assignment Problem," Discrete Applied Math., vol. 18, pp. 211-225, 1987.
[45] B. Monien, S. Tschvke, and R. Llling, "Solving the Traveling Sales-man Problem with a Distributed Branch-and-Bound Algorithm on a 1,024 Processor Network," Proc. Ninth Int'l Parallel Processing Symp., pp. 182-189,Santa Barbara, Calif., Apr. 1995.
[46] B.W. Wah, G.-J. Li, and C.F. Yu, "Multiprocessing of Combinatorial Search Problems," Computer, June 1985.
[47] B.W. Wah and Y.W.E. Ma, "MANIP—A Multicomputer Architecture for Solving Combinatorial Extremum-Search Problems," IEEE Trans. Computers, vol. 33, May 1984.

Index Terms:
Parallel processing, discrete optimization, tree search, load balancing, backtracking, heuristic search, game-tree search, speedup anomalies.
Ananth Grama, Vipin Kumar, "State of the Art in Parallel Search Techniques for Discrete Optimization Problems," IEEE Transactions on Knowledge and Data Engineering, vol. 11, no. 1, pp. 28-35, Jan.-Feb. 1999, doi:10.1109/69.755612
Usage of this product signifies your acceptance of the Terms of Use.