2006 15th IEEE International Conference on High Performance Distributed Computing
Runtime Support for Memory Adaptation in Scientific Applications via Local Disk and Remote Memory
Paris
June 19-June 23
ISBN: 1-4244-0307-3
The ever increasing memory demands of many scientific applications and the complexity of today's shared computational resources still require the occasional use of virtual memory, network memory, or even out-of-core implementations, with well known drawbacks in performance and usability. In this paper, we present a general framework, based on our earlier MML B prototype, that enables fully customizable, memory malleability in a wide variety of scientific applications. We provide several necessary enhancements to the environment sensing capabilities of MMLIB and introduce a remote memory capability, based on MPI communication of cached memory blocks between `compute nodes' and designated memory servers. We show experimental results from three important scientific applications that require the general MML B framework. Under constant memory pressure, we observe execution time improvements of factors between three and five over relying solely on the virtual memory system. With remote memory employed, these factors are even larger and significantly better than other, system-level remote memory implementations
Index Terms:
memory server, memory adaptation, scientific application, local disk memory, remote memory capability, shared computational resource, virtual memory system, network memory, memory malleability, MPI communication, cache memory
Citation:
C. Yue, R.T. Mills, A. Stathopoulos, D. Nikolopoulos, "Runtime Support for Memory Adaptation in Scientific Applications via Local Disk and Remote Memory," hpdc, pp.183-194, 2006 15th IEEE International Conference on High Performance Distributed Computing, 2006