The Community for Technology Leaders
2015 IEEE 31st International Conference on Data Engineering (ICDE) (2015)
Seoul, South Korea
April 13, 2015 to April 17, 2015
ISBN: 978-1-4799-7964-6
pp: 1328-1339
Ioannis Koltsidas , IBM Research - Zurich, Switzerland
Slavisa Sarafijanovic , IBM Research - Zurich, Switzerland
Martin Petermann , IBM Research - Zurich, Switzerland
Nils Haustein , IBM Systems and Technology Group, Mainz, Germany
Harald Seipp , IBM Systems and Technology Group, Mainz, Germany
Robert Haas , IBM Research - Zurich, Switzerland
Jens Jelitto , IBM Research - Zurich, Switzerland
Thomas Weigold , IBM Research - Zurich, Switzerland
Edwin Childers , IBM Systems and Technology Group, Tucson, Arizona, USA
David Pease , IBM Research - Almaden, San Jose, California, USA
Evangelos Eleftheriou , IBM Research - Zurich, Switzerland
ABSTRACT
The explosion of data volumes in enterprise environments and limited budgets have triggered the need for multi-tiered storage systems. With the bulk of the data being extremely infrequently accessed, tape is a natural fit for storing such data. In this paper we present our approach to a file storage system that seamlessly integrates disk and tape, enabling a bottomless and cost-effective storage architecture that can scale to accommodate Big Data requirements. The proposed system offers access to data through a POSIX filesystem interface under a single global namespace, optimizing the placement of data across disk and tape tiers. Using a self-contained, standardized and open filesystem format on the removable tape media, the proposed system avoids dependence on proprietary software and external metadata servers to access the data stored on tape. By internally managing the tape tier resources, such as tape drives and cartridges, the system relieves the user from the burden of dealing with the complexities of tape storage. Our implementation, which is based on the GPFS and LTFS filesystems, demonstrates the applicability of the proposed architecture in real-world environments. Our experimental evaluation has shown that this is a very promising approach in terms scalability, performance and manageability. The proposed system has been productized by IBM as LTFS Enterprise Edition.
INDEX TERMS
Libraries, Computer architecture, Standards, Indexes, File systems, Data transfer
CITATION
Ioannis Koltsidas, Slavisa Sarafijanovic, Martin Petermann, Nils Haustein, Harald Seipp, Robert Haas, Jens Jelitto, Thomas Weigold, Edwin Childers, David Pease, Evangelos Eleftheriou, "Seamlessly integrating disk and tape in a multi-tiered distributed file system", 2015 IEEE 31st International Conference on Data Engineering (ICDE), vol. 00, no. , pp. 1328-1339, 2015, doi:10.1109/ICDE.2015.7113380
93 ms
(Ver 3.3 (11022016))