The Community for Technology Leaders
2013 IEEE International Conference on Cluster Computing (CLUSTER) (2011)
Austin, Texas USA
Sept. 26, 2011 to Sept. 30, 2011
ISBN: 978-0-7695-4516-5
pp: 438-445
While network bandwidth is steadily increasing, it is doing so at a much slower rate than the corresponding increase in CPU performance. This trend has widened the gap between CPU and network speed. In this paper, we investigate improvements to I/O performance by exploiting this gap. We harness idle CPU resources to compress network traffic, reducing the amount of data transferred over the network and increasing effective network bandwidth. We created a set of compression services within the I/O Forwarding Scalability Layer. These services transparently compress and decompress data as it is transferred over the network. We studied the effect of the compression services on a variety of data sets and conducted experiments on a high-performance computing cluster.
I/O Forwarding, Scientific Data Compression
Dries Kimpe, Christina M. Patrick, Kamil Iskra, Jason Cope, Benjamin Welton, Robert Ross, "Improving I/O Forwarding Throughput with Data Compression", 2013 IEEE International Conference on Cluster Computing (CLUSTER), vol. 00, no. , pp. 438-445, 2011, doi:10.1109/CLUSTER.2011.80
97 ms
(Ver 3.3 (11022016))