The Community for Technology Leaders
2014 IEEE International Conference On Cluster Computing (CLUSTER) (2014)
Madrid, Spain
Sept. 22, 2014 to Sept. 26, 2014
ISBN: 978-1-4799-5548-0
pp: 113-122
Florian Klein , Institut für Informatik, Heinrich-Heine-Universität Düsseldorf, Universitätsstr. 1, 40225, Germany
Kevin Beineke , Institut für Informatik, Heinrich-Heine-Universität Düsseldorf, Universitätsstr. 1, 40225, Germany
Michael Schottner , Institut für Informatik, Heinrich-Heine-Universität Düsseldorf, Universitätsstr. 1, 40225, Germany
ABSTRACT
Large-scale interactive applications and online analytic processing on graphs require fast data access to huge sets of small data objects. DXRAM addresses these challenges by keeping all data always in memory of potentially many nodes aggregated in a data center. In this paper we focus on the efficient memory management and mapping of global IDs to local memory addresses, which is not trivial as each node may store up to one billion of small data objects (16–64 byte) in its local memory. We present an efficient paging-like translation scheme for global IDs and a memory management optimized for many small data objects. The latter includes an efficient incremental defragmentation supporting changing allocation granularities for dynamic data. Our evaluations show that the proposed memory management approach has only a 4–5% overhead compared to state of the art memory allocators with around 20% and the paging-like mapping of globals IDs is faster and more efficient than hash-table based approaches. Furthermore, we compare memory overhead and read performance of DXRAM with RAMCloud.
INDEX TERMS
Random access memory, Memory management, Peer-to-peer computing, Resource management, Java, Distributed databases, Indexes
CITATION

F. Klein, K. Beineke and M. Schottner, "Memory management for billions of small objects in a distributed in-memory storage," 2014 IEEE International Conference On Cluster Computing (CLUSTER), Madrid, Spain, 2014, pp. 113-122.
doi:10.1109/CLUSTER.2014.6968771
89 ms
(Ver 3.3 (11022016))