This Article 
 Bibliographic References 
 Add to: 
Adaptive On-the-Fly Compression
January 2006 (vol. 17 no. 1)
pp. 15-24

Abstract—We present a system called the Adaptive Compression Environment (ACE) that automatically and transparently applies compression (on-the-fly) to a communication stream to improve network transfer performance. ACE uses a series of estimation techniques to make short-term forecasts of compressed and uncompressed transfer time at the 32KB block level. ACE considers underlying networking technology, available resource performance, and data characteristics as part of its estimations to determine which compression algorithm to apply (if any). Our empirical evaluation shows that, on average, ACE improves transfer performance given changing network types and performance characteristics by 8 to 93 percent over using the popular compression techniques that we studied (Bzip, Zlib, LZO, and no compression) alone.

[1] A. Adl-Tabatabai, M. Cierniak, G. Lueh, V. Parikh, and J. Stichnoth, “Fast, Effective Code Generation in a Just-in-Time Java Compiler,” Proc. ACM SIGPLAN '98 Conf. Programming Language Design and Implementation, Oct. 2000.
[2] F. Berman, G. Fox, and T. Hey, Grid Computing: Making the Global Infrastructure a Reality. Wiley and Sons, 2003.
[3] BZIP Compression, http://sources.redhat combzip2/, 2005.
[4] Calgary corpus, http://links.uwaterloo.cacalgary.corpus.html+ , 2005.
[5] Canterbury corpus,, 2005.
[6] M. Cierniak, G. Lueh, and J. Stichnoth, “Practicing JUDO: Java under Dynamic Optimizations,” Proc. ACM SIGPLAN 2000 Conf. Programming Language Design and Implementation, Oct. 2000.
[7] Microsoft Corp., Microsoft.Net,, 2005.
[8] A. DeWitt, T. Gross, B. Lowekamp, N. Miller, P. Steenkiste, J. Subhlok, and D. Sutherland, “ReMoS: A Resource Monitoring System for Network-Aware Applications,” Technical Report CMU-CS-97-194, Dept. of Computer Science, Carnegie-Mellon Univ., Dec. 1998.
[9] I. Foster and C. Kesselman, The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers, Inc., 1998.
[10] Gnutella,, 2005.
[11] Grid Portal Collaboration, ngridproxynovotny/, 2005.
[12] Gzip homepage,, 2005.
[13] N. Hu, “Network Aware Data Transmission with Compression,” Technical Report CMU-CS-01-164, Dept. of Computer Science, Carnegie-Mellon Univ., 2001.
[14] Sun Microsystems Inc., The Java ARchive Utility, tooldocs/solarisjar.html, 2005.
[15] E. Jeannot, B. Knutsson, and M. Björkman, “Adaptive Online Data Compression,” Proc. IEEE Int'l Symp. High Performance Distributed Computing '02, July 2002.
[16] B. Knutsson and M. Bjorkman, “Adaptive End-To-End Compression for Variable-Bandwidth Communication,” Computer Networks, vol. 31, no. 7, pp. 767-779, Apr. 1999.
[17] C. Krintz and B. Calder, “Reducing Transfer Delay with Dyanamic Selection of Wire-Transfer Formats,” Proc. IEEE Int'l Symp. High Performance Distributed Computing (HPDC), Aug. 2001.
[18] Lempel-Ziv-Oberhumer (LZO) Compression, 2005,
[19] N. Motgi and A. Mukherjee, “Network Conscious Text Compression System (NCTCSys),” Proc. Int'l Conf. Information Technology: Coding and Computing, Apr. 2001.
[20] W. Pugh, “Compressing Java Class Files,” Proc. SIGPLAN '99 Conf. Programming Language Design and Implementation, May 1999.
[21] P. Sevcik, “Internet Bandwidth: It's Time for Accountability,” Business Comm. Rev., vol. 31, no. 1, pp. 1-3, Jan. 2001.
[22] S. Sucu and C. Krintz, “ACE: A Resource-Aware Adaptive Compression Environment,” Proc. Int'l Conf. Information Technology: Coding and Computing (ITCC '03), Apr. 2003.
[23] R. Wolski, “Dynamically Forecasting Network Performance Using the Network Weather Service,” Cluster Computing, 1998.
[24] R. Wolski, “Experiences with Predicting Resource Performance On-Line in Computational Grid Settings,” ACM SIGMETRICS Performance Evaluation Rev., vol. 30, no. 4, pp. 41-49, Mar. 2003.
[25] R. Wolski, N. Spring, and J. Hayes, “The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing,” Future Generation Computer Systems, 1999.
[26] ZLib compression library,, 2005.

Index Terms:
Adaptive compression, dynamic, performance prediction, mobile systems.
Chandra Krintz, Sezgin Sucu, "Adaptive On-the-Fly Compression," IEEE Transactions on Parallel and Distributed Systems, vol. 17, no. 1, pp. 15-24, Jan. 2006, doi:10.1109/TPDS.2006.3
Usage of this product signifies your acceptance of the Terms of Use.