This Article 
 Bibliographic References 
 Add to: 
An Application-Centric Characterization of Domain-Based SFC Partitioners for Parallel SAMR
December 2002 (vol. 13 no. 12)
pp. 1275-1289

Abstract—Structured adaptive mesh refinement (SAMR) methods for the numerical solution of partial differential equations yield highly advantageous ratios for cost/accuracy as compared to methods based on static uniform approximations. These techniques are being effectively used in many domains including computational fluid dynamics, numerical relativity, astrophysics, subsurface modeling, and oil reservoir simulation. Distributed implementations of these methods, however, lead to significant challenges in dynamic data-distribution, load-balancing, and runtime management. This paper presents an application-centric characterization of a suite of dynamic domain-based inverse space-filling curve partitioning techniques for the distributed adaptive grid hierarchies that underlie SAMR applications. The overall goal of this research is to formulate policies required to drive a dynamically adaptive metapartitioner for SAMR grid hierarchies capable of selecting the most appropriate partitioning strategy at runtime based on current application and system state. Such a metapartitioner can significantly reduce the execution time of SAMR applications.

[1] ASCI/ASAP Center, http://www.cacr.caltech.eduASAP, California Inst. of Tech nology. 2002.
[2] ASCI Alliance, , Univ. of Chicago. 2002.
[3] S. Baden, S. Kohn, and S. Fink, “Programming with LPARX,” technical report, Univ. of California, San Diego, 1994.
[4] M. Berger and S. Bokhari, “A Partitioning Strategy for Non-Uniform Problems on Multiprocessors,” IEEE Trans. Computers, vol. 36, pp. 570-580, 1987.
[5] M. Berger and P. Colella, “Local Adaptive Mesh Refinement for Shock Hydrodynamics,” J. Computational Physics, vol. 82, pp. 64-84, 1989.
[6] M. Berger, G. Hedstrom, J. Oliger, and G. Rodrigue, “Adaptive Mesh Refinement for 1-Dimensional Gas Dynamics,” Scientific Computing, pp. 43-47, 1983.
[7] M. Berger and J. Oliger, “Adaptive Mesh Refinement for Hyperbolic Partial Differential Equations,” J. Computational Physics, vol. 53, pp. 484-512, 1984.
[8] T. Bially, “A Class of Dimension Changing Mappings and Its Application to Bandwidth Compression,” PhD thesis, Polytechnic Inst. of Brooklyn, 1967.
[9] S. Bokhari, T. Crockett, and D. Nicol, “Binary Dissection: Variants and Applications,” Technical Report ICASE Report No. 97-29, NASA Langley Research Center, Hampton, Va., 1997.
[10] G. Bryan, “Fluids in the Universe: Adaptive Mesh Refinement in Cosmology,” Computing in Science and Eng., pp. 46-53, Mar. 1999.
[11] A. Calder and B. Curtis, “High Performance Reactive Fluid Flow Simulations Using Adaptive Mesh Refinement on Thousands of Processors,” Proc. Supercomputing Conf., 2000.
[12] S. Chandra and M. Parashar, “An Evaluation of Partitioners for Parallel SAMR Applications,” Euro-Par 2001, Springer-Verlag Lecture Notes in Computer Science, eds., R. Sakellariou, J. Keane, J. Gurd, and L. Freeman, vol. 2150, pp. 171-174, Aug. 2001.
[13] S. Chandra, J. Steensland, and M. Parashar, “An Experimental Study of Adaptive Application Sensitive Partitioning Strategies for SAMR Applications,” submitted for poster presentation at Supercomputing Conf., 2001.
[14] M. Choptuik, “Experiences with an Adaptive Mesh Refinement Algorithm in Numerical Relativity,” Frontiers in Numerical Relativity, London Cambridge Univ. Press, eds., C. Evans, L. Finn, and D. Hobill, pp. 206-221, 1989.
[15] C. Edwards, “A Parallel Infrastructure for Scalable Adaptive Finite Element Methods and Its Application to Least Squares C-Infinity Collocation,” PhD thesis, Univ. of Texas at Austin, May 1997.
[16] S. Fink, S. Baden, and S. Kohn, “Flexible Communication Mechanisms for Dynamic Structured Applications,” IRREGULAR, 1996.
[17] M. Griebel and G. Zumbusch, “Hash-Storage Techniques for Adaptive Multilevel Solvers and Their Domain Decomposition Parallelization,” Domain Decomposition Methods 10, J. Mandel, C. Farhat, and X. Cai, eds., pp. 279-286, 1998.
[18] M. Griebel and G. Zumbusch, “Parallel Multigrid in an Adaptive PDE Solver Based on Hashing,” Proc. ParCo '97, E. D'Hollander, G. Joubert, F. Peters, and U. Trottenberg, eds., pp. 589-599, 1998.
[19] S. Hawley and M. Choptuik, “Boson Stars Driven to the Brink of Black Hole Formation,” Physics Review D vol. 62, no. 104024, 2000.
[20] S. Kohn, “SAMRAI: Structured Adaptive Mesh Refinement Applications Infrastructure,”, 1999.
[21] S. Kohn and S. Baden, “Parallel Software Abstractions for Structured Adaptive Mesh Methods,” J. Parallel and Distributed Computing, vol. 61, no. 6, pp. 713-736, 2001.
[22] P. MacNeice Paramesh homepage, parameshparamesh.html, 1999.
[23] B. Moon, H. Jagadish, C. Faloutsos, and J. Saltz, “Analysis of the Clustering Properties of Hilbert Space-Filling Curve,” IEEE Trans. Knowledge and Data Engineering, vol. 13, no. 1, Jan./Feb. 2001, also Univ. of Maryland Dept. of Computer Science Technical Report, CS-TR-3611 and UMIACS-TR-96-20, Mar. 1996,
[24] M. Norman and G. Bryan, “Cosmological Adaptive Mesh Refinement,” Numerical Astrophysics 1998, Dordrecht: The Netherlands: Kluwer Academic, 1999.
[25] L. Oliker and R. Biswas, “PLUM: Parallel Load Balancing for Adaptive Unstructured Meshes,” J. Parallel and Distributed Computing, vol. 52, no. 2, pp. 150-177, 1998.
[26] B. Olstad and F. Manne, “Efficient Partitioning of Sequences,” IEEE Trans. Computers, vol. 44, no. 11, Nov. 1995.
[27] C. Ou and S. Ranka, “Parallel Remapping Algorithms for Adaptive Problems,” J. Parallel and Distributed Computing, vol. 42, pp. 109-121, 1997.
[28] M. Parashar and J. Browne, “On Partitioning Dynamic Adaptive Grid Hierarchies,” Proc. 29th Ann. Hawaii Int'l Conf. System Sciences, pp. 604-613, Jan. 1996.
[29] M. Parashar, J. Browne, C. Edwards, and K. Klimkowski, “A Common Data Management Infrastructure for Adaptive Algorithms for PDE Solutions,” Proc. Supercomputing Conf., 1997.
[30] M. Parashar and J. Browne, “System Engineering for High Performance Computing Software: The HDDA/DAGH Infrastructure for Implementation of Parallel Structured Adaptive Mesh Refinement,” IMA Volume on Structured Adaptive Mesh Refinement (SAMR) Grid Methods, S. Baden, N. Chrisochoides, D. Gannon, and M. Norman, eds., pp. 1-18, Springer-Verlag, Jan. 2000.
[31] M. Parashar, J. Wheeler, G. Pope, K. Wang, and P. Wang, “A New Generation EOS Compositional Reservoir Simulator: Part II-Framework and Multiprocessing,” Proc. Soc. Petroleum Eng. Reservoir Simulation Symp., June 1997.
[32] A. Patra, J. Long, and A. Laszloffy, “AFEAPI: A Simple Infrastructure for Parallel Adaptive hp FEM Using Self-Organizing Data and Computations,” Proc. First Workshop Parallel Computing for Irregular Applications, Jan. 1999.
[33] R. Pember, J. Bell, P. Colella, W. Crutchfield, and M. Welcome, “Adaptive Cartesian Grid Methods for Representing Geometry in Inviscid Compressible Flow,” Proc. 11th AIAA Computational Fluid Dynamics Conf., July 1993.
[34] J. Pilkington and S. Baden, “Dynamic Partitioning of Non-Uniform Structured Workloads with Spacefilling Curves,” IEEE Trans. Parallel and Distributed Systems, vol. 7, no. 3, pp. 288-300, Mar. 1996.
[35] J. Pilkington and S. Baden, “Partitioning with Space-Filling Curves,” Technical Report CS94-34, Univ. of California, San Diego, 1994.
[36] J. Rantakokko, “Data Partitioning Methods and Parallel Block-Oriented PDE Solvers,” PhD thesis, Uppsala Univ., Sweden, 1998.
[37] H. Sagan, Space-filling Curves. Springer-Verlag, 1994.
[38] J. Saltzman, M. Berger, J. Bell, and M. Welcome, “Three Dimensional Adaptive Mesh Refinement for Hyperbolic Conservation Laws,” SIAM J. Scientific Computing, vol. 15, pp. 127-138, 1994.
[39] H. Samet, The Design and Analysis of Spatial Data Structures. Reading, Mass: Addison-Wesley, 1989.
[40] M. Shee, “Evaluation and Optimization of Load Balancing/Distribution Techniques for Adaptive Grid Hierarchies,” MS thesis, Graduate School, Rutgers Univ., N.J., 2000.
[41] J. Steensland Vampire homepage, , 2000.
[42] J. Steensland, “Domain-Based Partitioning for Parallel SAMR Applications,” Licentiate thesis, 2001-2002, Dept. of Scientific Computing, Uppsala Univ., Sweden, 2001.
[43] J. Steensland, S. Chandra, M. Thuné, and M. Parashar, “Characterization of Domain-Based Partitioners for Parallel SAMR Applications,” Proc. IASTED Int'l Conf. Parallel and Distributed Computing and Systems, pp. 425-430, Nov. 2000.
[44] J. Steensland, S. Soderberg, and M. Thuné, A Comparison of Partitioning Schemes for Blockwise Parallel SAMR Applications. Springer-Verlag, 2000.
[45] E. Steinthorsson and D. Modiano, “Advanced Methodology for Simulation of Complex Flows Using Structured Grid Systems,” Proc. Inst. for Computational Mechanics in Propulsion, vol. 28, 1995.
[46] M. Thuné, “Partitioning Strategies for Composite Grids,” Parallel Algorithms and Applications, 1997.
[47] N. Touheed, P. Selwood, P. Jimack, and M. Berzins, “A Comparison of Some Dynamic Load-Balancing Algorithms for a Parallel Adaptive Flow Solver,” J. Parallel Computing, vol. 26, pp. 1535-1554, 2000.
[48] C. Walshaw, M. Cross, and M. Everett, “Parallel Dynamic Graph Partitioning for Adaptive Unstructured Meshes,” J. Parallel and Distributed Computing, vol. 47, pp. 102-108, 1997.
[49] M. Warren and J. Salmon, “A Parallel Hashed Oct-Tree N-Body Algorithm,” Proc. Supercomputing Conf., pp. 12-21, 1993.
[50] P. Wang, I. Yotov, T. Arbogast, C. Dawson, M. Parashar, and K. Sepehrnoori, “A New Generation EOS Compositional Reservoir Simulator: Part I-Formulation and Discretization,” Proc. Soc. Petroleum Eng. Reservoir Simulation Symp., June 1997.

Index Terms:
Dynamic domain-based partitioning and load-balancing, inverse space-filling curve partitioning, performance characterization, structured adaptive mesh refinement, adaptive metapartitioner.
Johan Steensland, Sumir Chandra, Manish Parashar, "An Application-Centric Characterization of Domain-Based SFC Partitioners for Parallel SAMR," IEEE Transactions on Parallel and Distributed Systems, vol. 13, no. 12, pp. 1275-1289, Dec. 2002, doi:10.1109/TPDS.2002.1158265
Usage of this product signifies your acceptance of the Terms of Use.