Implementation and performance of a parallel file system for high performance distributed applications
High-Performance Distributed Computing, International Symposium on (1996)
Syracuse, New York
Aug. 6, 1996 to Aug. 9, 1996
W.B. Ligon, III , Clemson Univ., SC, USA
R.B. Ross , Clemson Univ., SC, USA
Dedicated cluster parallel computers (DCPCs) are emerging as low-cost high performance environments for many important applications in science and engineering. A significant class of applications that perform well on a DCPC are coarse-grain applications that involve large amounts of file I/O. Current research in parallel file systems for distributed systems is providing a mechanism for adapting these applications to the DCPC environment. We present the Parallel Virtual File System (PVFS), a system that provides disk striping across multiple nodes in a distributed parallel computer and file partitioning among tasks in a parallel program. PVFS is unique among similar systems in that it uses a stream-based approach that represents each file access with a single set of request parameters and decouples the number of network messages from details of the file striping and partitioning. PVFS also provides support for efficient collective file accesses and allows overlapping file partitions. We present results of early performance experiments that show PVFS achieves excellent speedups in accessing moderately sized file segments.
parallel programming; file organisation; software performance evaluation; parallel machines; message passing; parallel file system performance; high performance distributed applications; dedicated cluster parallel computers; low-cost high performance environments; science; engineering; coarse-grain applications; file input output; Parallel Virtual File System; PVFS; disk striping; file partitioning; parallel program; stream-based approach; file access; network messages; file striping
W. Ligon, III and R. Ross, "Implementation and performance of a parallel file system for high performance distributed applications," High-Performance Distributed Computing, International Symposium on(HPDC), Syracuse, New York, 1996, pp. 471.