20th IEEE International Conference on Distributed Computing Systems (ICDCS'00)
Prediction-Capable Data Compression Algorithms for Improving Transmission Efficiency on Distributed Systems
Taipei, Taiwan
April 10-April 13
ISBN: 0-7695-0601-1
Network bandwidth is a kind of limited and precious resources in modern distributed computing environments. Insufficient bandwidth will severely degrade the performance of a distributed computing task in exchanging massive data among the networked hosts. A feasible solution to save bandwidth is to incorporate data compression during transmission time. However, blind or unconditional compression may only result in waste of CPU power and even slow down the overall network transfer rate, if the data to transmit are hard to compress.In this paper we present a prediction-capable lossless data compression algorithm to address this problem. By adapting to the compression speed of a host CPU, current system load, and network speed, our algorithm can accurately estimate the compression time of each data block given, and decide whether it should be compressed or not. Experimental results indicate that our prediction mechanism is both efficient and effective, achieving 93% of prediction accuracy at the cost of only 3.2% of the execution time of unconditional compression.
Citation:
Hann-Huei Chiou, Alexander I-Chi Lai, Chin-Laung Lei, "Prediction-Capable Data Compression Algorithms for Improving Transmission Efficiency on Distributed Systems," icdcs, pp.654, 20th IEEE International Conference on Distributed Computing Systems (ICDCS'00), 2000