2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS) (2016)
Wuhan, Hubei, China
Dec. 13, 2016 to Dec. 16, 2016
Current shared cloud storage cannot provide sufficient I/O throughput for data-intensive HPC applications. Moreover, the consistency policy used in most shared cloud storage can cause parallel I/O applications to fail due to unexpected file inconsistencies. In order to resolve these problems, we propose a novel fast, scalable and fault tolerant filesystem called CloudBB (Cloud-based Burst Buffer). Unlike conventional filesystems, CloudBB creates an on-demand two-level hierarchical storage system and caches popular files to accelerate I/O performance. Since CloudBB supports multiple metadata servers, CloudBB is also highly scalable. In addition, by using file replication, failure detection and recovery techniques, CloudBB is resilient to failures. Furthermore, we implement CloudBB by using FUSE so that existing applications can run seamlessly and benefit from all of the CloudBB's capabilities without code modification. To validate the effectiveness of CloudBB, we evaluate performance of real data-intensive HPC applications in Amazon EC2/S3. The results show CloudBB improves performance by up to 28.7 times while reducing cost by up to 94.7% compared to the ones without CloudBB.
Cloud computing, Ions, Buffer storage, Metadata, Computer architecture, Throughput, Acceleration
T. Xu, K. Sato and S. Matsuoka, "CloudBB: Scalable I/O Accelerator for Shared Cloud Storage," 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS), Wuhan, Hubei, China, 2016, pp. 509-518.