This Article 
 Bibliographic References 
 Add to: 
Hybrid Runtime Management of Space-Time Heterogeneity for Parallel Structured Adaptive Applications
September 2007 (vol. 18 no. 9)
pp. 1202-1214
Structured adaptive mesh refinement (SAMR) techniques provide an effective means for dynamically concentrating computational effort and resources to appropriate regions in the application domain. However, due to their dynamism and space-time heterogeneity, scalable parallel implementation of SAMR applications remains a challenge. This paper investigates hybrid runtime management strategies and presents an adaptive hierarchical multi-partitioner (AHMP) framework. AHMP dynamically applies multiple partitioners to different regions of the domain, in a hierarchical manner, to match the local requirements of the regions. Key components of the AHMP framework include a segmentation-based clustering algorithm (SBC) that can efficiently identify regions in the domain with relatively homogeneous partitioning requirements, mechanisms for characterizing the partitioning requirements of these regions, and a runtime system for selecting, configuring and applying the most appropriate partitioner to each region. Further, to address dynamic resource situations for long running applications, AHMP provides a hybrid partitioning strategy (HPS), which involves application-level pipelining, trading space for time when resources are sufficiently large and under-utilized, and an application-level out-of-core strategy (ALOC), trading time for space when resources are scarce in order to enhance the survivability of applications. The AHMP framework has been implemented and experimentally evaluated on up to 1280 processors of the IBM SP4 cluster at San Diego Supercomputer Center.

[1] M. Berger and J. Oliger, “Adaptive Mesh Refinement for Hyperbolic Partial Differential Equations,” J. Computational Physics, vol. 53, pp. 484-512, 1984.
[2] M. Berger and P. Colella, “Local Adaptive Mesh Refinement for Shock Hydrodynamics,” J. Computational Physics, vol. 82, pp. 64-84, 1989.
[3] K. Devine, E. Boman, R. Heaphy, B. Hendrickson, and C. Vaughan, “Zoltan Data Management Services for Parallel Dynamic Applications,” Computing in Science and Eng., vol. 4, no. 2, pp. 90-97, Mar.-Apr. 2002.
[4] S. Das, D. Harvey, and R. Biswas, “Parallel Processing of Adaptive Meshes with Load Balancing,” IEEE Tran. Parallel and Distributed Systems, vol. 12, no. 12, pp. 1269-1280, Dec. 2001.
[5] J. Cummings, M. Aivazis, R. Samtaney, R. Radovitzky, S. Mauch, and D. Meiron, “A Virtual Test Facility for the Simulation of Dynamic Response in Materials,” J. Supercomputing, vol. 23, pp.39-50, 2002.
[6] S. Hawley and M. Choptuik, “Boson Stars Driven to the Brink of Black Hole Formation,” Physical Rev. D, vol. 62, no. 10, p. 104024, 2000.
[7] J. Ray, H.N. Najm, R.B. Milne, K.D. Devine, and S. Kempka, “Triple Flame Structure and Dynamics at the Stabilization Point of an Unsteady Lifted Jet Diffusion Flame,” Proc. Combustion Inst., vol. 25, no. 1, pp. 219-226, 2000.
[8] A. Calder, H. Tufo, J. Turan, M. Zingale, G. Henry, B. Curtis, L. Dursi, B. Fryxell, P. MacNeice, K. Olson, P. Ricker, R. Rosner, and F. Timmes, “High Performance Reactive Fluid Flow Simulations Using Adaptive Mesh Refinement on Thousands of Processors,” Proc. ACM/IEEE Conf. Supercomputing, 2000.
[9] L.V. Kale, “Charm,”, 2007.
[10] R.D. Hornung and S.R. Kohn, “Managing Application Complexity in the SAMRAI Object-Oriented Framework,” Concurrency and Computation—Practice & Experience, vol. 14, no. 5, pp. 347-368, 2002.
[11] G. Karypis and V. Kumar, “Parmetis,” http://www.users.cs.umn. edu/~karypis/metis/ parmetisindex.html, 2003.
[12] P. MacNeice, “Paramesh,” parameshparamesh.html, 2007
[13] M. Parashar and J. Browne, “On Partitioning Dynamic Adaptive Grid Hierarchies,” Proc. 29th Ann. Hawaii Int'l Conf. System Sciences, pp.604-613, 1996.
[14] J. Steensland, S. Chandra, and M. Parashar, “An Application-Centric Characterization of Domain-Based SFC Partitioners for Parallel SAMR,” IEEE Trans. Parallel and Distributed Systems, vol. 13, no. 12, pp. 1275-1289, Dec. 2002.
[15] P.E. Crandall and M.J. Quinn, “A Partitioning Advisory System for Networked Data-Parallel Programming,” Concurrency: Practice and Experience, vol. 7, no. 5, pp. 479-495, 1995.
[16] X. Li and M. Parashar, “Dynamic Load Partitioning Strategies for Managing Data of Space and Time Heterogeneity in Parallel SAMR Applications,” Proc. Ninth Int'l Euro-Par Conf. (Euro-Par '03), vol. 2790, pp.181-188, 2003.
[17] X. Li and M. Parashar, “Using Clustering to Address the Heterogeneity and Dynamism in Parallel SAMR Application,” Proc. 12th Ann. IEEE Int'l Conf. High Performance Computing (HiPC '05), 2005.
[18] A. Wissink, D. Hysom, and R. Hornung, “Enhancing Scalability of Parallel Structured AMR Calculations,” Proc. 17th ACM Int'l Conf. Supercomputing (ICS '03), pp. 336-347, 2003.
[19] M. Thune, “Partitioning Strategies for Composite Grids,” Parallel Algorithms and Applications, vol. 11, pp. 325-348, 1997.
[20] Z. Lan, V. Taylor, and G. Bryan, “Dynamic Load Balancing for Adaptive Mesh Refinement Applications: Improvements and Sensitivity Analysis,” Proc. 13th IASTED Int'l Conf. Parallel and Distributed Computing Systems (PDCS '01), 2001.
[21] Z. Lan, V. Taylor, and G. Bryan, “Dynamic Load Balancing of SAMR Applications on Distributed Systems,” J. Scientific Programming, vol. 10, no. 4, pp. 319-328, 2002.
[22] M. Berger and I. Regoutsos, “An Algorithm for Point Clustering and Grid Generation,” IEEE Trans. Systems, Man, and Cybernetics, vol. 21, no. 5, pp. 1278-1286, 1991.
[23] J.D. Teresco, J. Faik, and J.E. Flaherty, “Hierarchical Partitioning and Dynamic Load Balancing for Scientific Computation,” Technical Report CS-04-04, Dept. of Computer Science, Williams College, 2004, also in Proc. Workshop Applied Parallel Computing (PARA '04), 2004.
[24] J. Steensland, “Efficient Partitioning of Structured Dynamic Grid Hierarchies,” PhD dissertation, Uppsala Univ., 2002.
[25] H. Sagan, Space Filling Curves. Springer-Verlag, 1994.
[26] J. Pilkington and S. Baden, “Dynamic Partitioning of Non-Uniform Structured Workloads with Spacefilling Curves,” IEEE Trans. Parallel and Distributed Systems, vol. 7, no. 3, Mar. 1996.
[27] R.C. Gonzalez and R.E. Woods, Digital Image Processing, second ed. Prentice Hall, 2002.
[28] T. Ridler and S. Calvard, “Picture Thresholding Using an Iterative Selection Method,” IEEE Trans. Systems, Man, and Cybernetics, vol. 8, no. 630-632, 1978.
[29] N. Otsu, “A Threshold Selection Method from Gray-Level Histogram,” IEEE Trans. Systems, Man, and Cybernetics, vol. 6, no. 1, pp. 62-66, 1979.
[30] M. Parashar, “Grace,” /, 2007.
[31] X. Li and M. Parashar, “Hierarchical Partitioning Techniques for Structured Adaptive Mesh Refinement Applications,” J. Supercomputing, vol. 28, no. 3, pp. 265-278, 2004.
[32] J.L. Hennessy, D.A. Patterson, and D. Goldberg, Computer Architecture: A Quantitative Approach. Morgan Kaufmann, 2002.
[33] M.S. Potnuru, “Automatic Out-of-Core Execution Support for CHARM++,” master's thesis, Univ. of Illinois at Urbana-Champaign, 2003.
[34] N. Saboo and L.V. Kale, “Improving Paging Performance with Object Prefetching,” Proc. Int'l Conf. High Performance Computing (HiPC01), 2001.
[35] J. Tang, B. Fang, M. Hu, and H. Zhang, “Developing a User-Level Middleware for Out-of-Core Computation on Grids,” Proc. IEEE Int'l Symp. Cluster Computing and the Grid, pp. 686-690, 2004.
[36] “IPARS,” http://www.cpge.utexas.edunew_generation /, 2007.
[37] S. Chandra and M. Parashar, “A Simulation Framework for Evaluating the Runtime Characteristics of Structured Adaptive Mesh Refinement Applications,” Technical Report TR-275, Center for Advanced Information Processing, Rutgers Univ., Sept. 2004.
[38] “Hdf5,” http://hdf.ncsa.uiuc.eduHDF5/, 2007.

Index Terms:
Parallel Computing, Structured Adaptive Mesh Refinement, Dynamic Load Balancing, Hierarchical Multi-Partitioner, High Performance Computing
Xiaolin Li, Manish Parashar, "Hybrid Runtime Management of Space-Time Heterogeneity for Parallel Structured Adaptive Applications," IEEE Transactions on Parallel and Distributed Systems, vol. 18, no. 9, pp. 1202-1214, Sept. 2007, doi:10.1109/TPDS.2007.1038
Usage of this product signifies your acceptance of the Terms of Use.