2008 3rd Petascale Data Storage Workshop. (PDSW '08) (2008)
Nov. 17, 2008 to Nov. 17, 2008
Andrew W. Leung , Storage Systems Research Center, University of California, Santa Cruz, 95064, USA
Ethan L. Miller , Storage Systems Research Center, University of California, Santa Cruz, 95064, USA
As file system capacities reach the petascale, it is becoming increasingly difficult for users to organize, find, and manage their data. File system search has the potential to greatly improve how users manage and access files. Unfortunately, existing file system search is designed for smaller scale systems, making it difficult for existing solutions to scale to petascale files systems. In this paper, we motivate the importance of file system search in petascale file systems and present a new full text file system search design for petascale file systems. Unlike existing solutions, our design exploits file system properties. Using a novel index partitioning mechanism that utilizes file system namespace locality, we are able to improve search scalability and performance and we discuss how such a design can potentially improve search security and ranking.We describe how our design can be implemented within the Ceph petascale file system.
file organisation, information retrieval, text analysis
A. W. Leung and E. L. Miller, "Scalable full-text search for petascale file systems," 2008 3rd Petascale Data Storage Workshop. (PDSW '08)(PDSW), Austin, TX, 2009, pp. 1-7.