The Community for Technology Leaders
2015 IEEE 31st International Conference on Data Engineering (ICDE) (2015)
Seoul, South Korea
April 13, 2015 to April 17, 2015
ISBN: 978-1-4799-7964-6
pp: 1268-1279
Hao Zhang , School of Computing, National University of Singapore, Singapore
Gang Chen , College of Computer Science, Zhejiang University, China
Beng Chin Ooi , School of Computing, National University of Singapore, Singapore
Weng-Fai Wong , School of Computing, National University of Singapore, Singapore
Shensen Wu , yzBigData Co., Ltd., China
Yubin Xia , Institute of Parallel and Distributed Systems (IPADS), Shanghai Jiao Tong University, China
ABSTRACT
The increase in the capacity of main memory coupled with the decrease in cost has fueled the development of in-memory database systems that manage data entirely in memory, thereby eliminating the disk I/O bottleneck. However, as we shall explain, in the Big Data era, maintaining all data in memory is impossible, and even unnecessary. Ideally we would like to have the high access speed of memory, with the large capacity and low price of disk. This hinges on the ability to effectively utilize both the main memory and disk. In this paper, we analyze state-of-the-art approaches to achieving this goal for in-memory databases, which is called as “Anti-Caching” to distinguish it from traditional caching mechanisms. We conduct extensive experiments to study the effect of each fine-grained component of the entire process of “Anti-Caching” on both performance and prediction accuracy. To avoid the interference from other unrelated components of specific systems, we implement these approaches on a uniform platform to ensure a fair comparison. We also study the usability of each approach, and how intrusive it is to the systems that intend to incorporate it. Based on our findings, we propose some guidelines on designing a good “Anti-Caching” approach, and sketch a general and efficient approach, which can be utilized in most in-memory database systems without much code modification.
INDEX TERMS
Memory management, Big data, Linux, Semantics, Database systems, Real-time systems
CITATION
Hao Zhang, Gang Chen, Beng Chin Ooi, Weng-Fai Wong, Shensen Wu, Yubin Xia, "“Anti-Caching”-based elastic memory management for Big Data", 2015 IEEE 31st International Conference on Data Engineering (ICDE), vol. 00, no. , pp. 1268-1279, 2015, doi:10.1109/ICDE.2015.7113375
80 ms
(Ver 3.3 (11022016))