Issue No. 04 - April (2011 vol. 22)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPDS.2010.116
Yu Hua , Huazhong University of Science and Technology, Wuhan
Yifeng Zhu , University of Maine, Orono
Hong Jiang , University of Nebraska-Lincoln, Lincoln
Dan Feng , Huazhong University of Science and Technology, Wuhan
Lei Tian , Huazhong University of Science and Technology, Wuhan
This paper presents a scalable and adaptive decentralized metadata lookup scheme for ultralarge-scale file systems (more than Petabytes or even Exabytes). Our scheme logically organizes metadata servers (MDSs) into a multilayered query hierarchy and exploits grouped Bloom filters to efficiently route metadata requests to desired MDSs through the hierarchy. This metadata lookup scheme can be executed at the network or memory speed, without being bounded by the performance of slow disks. An effective workload balance method is also developed in this paper for server reconfigurations. This scheme is evaluated through extensive trace-driven simulations and a prototype implementation in Linux. Experimental results show that this scheme can significantly improve metadata management scalability and query efficiency in ultralarge-scale storage systems.
File systems, Bloom filters, metadata management, scalability, performance evaluation.
Y. Hua, Y. Zhu, D. Feng, H. Jiang and L. Tian, "Supporting Scalable and Adaptive Metadata Management in Ultralarge-Scale File Systems," in IEEE Transactions on Parallel & Distributed Systems, vol. 22, no. , pp. 580-593, 2010.