The Community for Technology Leaders
2013 IEEE 33rd International Conference on Distributed Computing Systems (2007)
Toronto, Canada
June 25, 2007 to June 27, 2007
ISBN: 0-7695-2837-3
pp: 14
Michael Kaminsky , Intel Research Pittsburgh
Srinivasan Seshan , Carnegie Mellon University
Jeffrey Pang , Carnegie Mellon University
Haifeng Yu , National University of Singapore
Phillip B. Gibbons , Intel Research Pittsburgh
ABSTRACT
Existing DHT-based file systems use consistent hashing to assign file blocks to random machines. As a result, a user task accessing an entire file or multiple files needs to retrieve blocks from many different machines. This paper demonstrates that significant availability and performance gains can be achieved if, instead, users are able to retrieve all the data needed for a given task from only a few DHT nodes. We explore the design and implications of such a "defragmented" DHT-based distributed file system, called D2, that also maintains important DHT properties like storage load balance. We show using real-world file system traces that a simple key encoding scheme is sufficient to maintain good defragmentation for most user tasks. Using both simulation and an actual 1,000 node deployment, we show that D2 increases availability by over an order of magnitude and improves user-perceived latency by 30" 100% compared to a traditional design.
INDEX TERMS
CITATION
Michael Kaminsky, Srinivasan Seshan, Jeffrey Pang, Haifeng Yu, Phillip B. Gibbons, "Defragmenting DHT-based Distributed File Systems", 2013 IEEE 33rd International Conference on Distributed Computing Systems, vol. 00, no. , pp. 14, 2007, doi:10.1109/ICDCS.2007.97
108 ms
(Ver )