The Community for Technology Leaders
RSS Icon
Issue No.01 - January (2009 vol.58)
pp: 18-31
Martin Burtscher , University of Texas at Austin, Austin
Paruj Ratanaworabhan , Cornell University, Ithaca
Many scientific programs exchange large quantities of double-precision data between processing nodes and with mass storage devices. Data compression can reduce the number of bytes that need to be transferred and stored. However, compression is only likely to be employed in high-end computing environments if it does not impede the throughput. This paper describes and evaluates FPC, a fast lossless compression algorithm for linear streams of 64-bit floating-point data. FPC works well on hard-to-compress scientific datasets and meets the throughput demands of high-performance systems. A comparison with five lossless compression schemes, BZIP2, DFCM, FSD, GZIP, and PLMI, on four architectures and thirteen datasets shows that FPC compresses and decompresses one to two orders of magnitude faster than the other algorithms at the same geometric-mean compression ratio. Moreover, FPC provides a guaranteed throughput as long as the prediction tables fit into the L1 data cache. For example, on a 1.6 GHz Itanium 2 server, the throughput is 670 megabytes per second regardless of what data are being compressed.
Data compaction and compression, Code tuning
Martin Burtscher, Paruj Ratanaworabhan, "FPC: A High-Speed Compressor for Double-Precision Floating-Point Data", IEEE Transactions on Computers, vol.58, no. 1, pp. 18-31, January 2009, doi:10.1109/TC.2008.131
[1] D. Bailey, T. Harris, W. Saphir, R.V.D. Wijngaart, A. Woo, and M. Yarrow, “The NAS Parallel Benchmarks 2.0,” Technical Report NAS-95-020, NASA Ames Research Center, 1995.
[2] M. Burrows and D.J. Wheeler, “A Block-Sorting Lossless Data Compression Algorithm,” Digital SRC Research Report 124, May 1994.
[3] M. Burtscher, “VPC3: A Fast and Effective Trace-Compression Algorithm,” Proc. Joint Int'l Conf. Measurement and Modeling of Computer Systems (MMCS '04), pp. 167-176, June 2004.
[4] M. Burtscher and P. Ratanaworabhan, “High Throughput Compression of Double-Precision Floating-Point Data,” Proc. Data Compression Conf. (DCC '07), pp. 293-302, Mar. 2007.
[5] M. Burtscher and I. Szczyrba, “Numerical Modeling of Brain Dynamics in Traumatic Situations—Impulsive Translations,” Proc. Int'l Conf. Math. and Eng. Techniques in Medicine and Biological Sciences (METMBS '05), pp. 205-211, June 2005.
[6] M. Burtscher and B.G. Zorn, “Exploring Last $n$ Value Prediction,” Proc. Int'l Conf. Parallel Architectures and Compilation Techniques (PACT '99), pp. 66-76, Oct. 1999.
[7] D. Chen, Y.-J. Chiang, and N. Memon, “Lossless Compression of Point-Based 3D Models,” Pacific Graphics, pp. 124-126, Oct. 2005.
[8] “Compiler Writer's Guide for the Alpha 21264,” Compaq, literature dsc-library.html , June 1999.
[9] V. Engelson, D. Fritzson, and P. Fritzson, “Lossless Compression of High-Volume Numerical Data from Simulations,” Proc. Data Compression Conf. (DCC '00), pp. 574-586, Mar. 2000.
[10] J. Fowler and R. Yagel, “Lossless Compression of Volume Data,” Proc. IEEE Symp. Volume Visualization (VV '94), pp. 43-50, 1994.
[11] F. Gabbay, “Speculative Execution Based on Value Prediction,” Technical Report #1080, Electrical Eng. Dept., Technion-Israel Inst. Tech nology, Nov. 1996.
[12] M.N. Gamito and M.S. Dias, “Lossless Coding of Floating Point Data with JPEG 2000 Part 10,” Proc. 27th Conf. Applications of Digital Image Processing (ADIP '04), pp. 276-287, 2004.
[13] F. Ghido, “An Efficient Algorithm for Lossless Compression of IEEE Float Audio,” Proc. Data Compression Conf. (DCC '04), pp.429-438, Mar. 2004.
[14] B. Goeman, H. Vandierendonck, and K. Bosschere, “Differential FCM: Increasing Value Prediction Accuracy by Improving Table Usage Efficiency,” Proc. Seventh Int'l Symp. High Performance Computer Architecture (HPCA '01), pp. 207-216, Jan. 2001.
[15] http:/, 2007.
[16] http:/, 2007.
[17] purple/rfp/benchmarks/limitedcode_list.html , 2007.
[18] L. Ibarria, P. Lindstrom, J. Rossignac, and A. Szymczak, “Out-of-Core Compression and Decompression of Large n-Dimensional Scalar Fields,” Proc. Eurographics '03, pp. 343-348, Sept. 2003.
[19] Intel Corporation, “Intel Itanium 2 Processor Reference Manual for Software Development and Optimization,” http://www. 251110.htm, May 2004.
[20] M. Isenburg, P. Lindstrom, and J. Snoeyink, “Lossless Compression of Floating-Point Geometry,” Proc. Computer Aided Design Conf. (CAD '04), pp. 495-502, 2004.
[21] J. Ke, M. Burtscher, and E. Speight, “Runtime Compression of MPI Messages to Improve the Performance and Scalability of Parallel Applications,” Proc. High-Performance Computing, Networking and Storage Conf. (SC '04), pp. 59-65, Nov. 2004.
[22] S. Klimenko, B. Mours, P. Shawhan, and A. Sazonov, “Data Compression Study with the E2 Data,” LIGO-T010033-00-E Technical Report, pp. 1-14, 2001.
[23] P. Lindstrom and M. Isenburg, “Fast and Efficient Compression of Floating-Point Data,” IEEE Trans. Visualization and Computer Graphics, vol. 12, no. 5, Sept./Oct. 2006.
[24] M.H. Lipasti, C.B. Wilkerson, and J.P. Shen, “Value Locality and Load Value Prediction,” Proc. Seventh Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS '96), pp. 138-147, Oct. 1996.
[25] J.M. Prusa, P.K. Smolarkiewicz, and A.A. Wyszogrodzki, “Simulations of Gravity Wave Induced Turbulence Using 512 PE CRAY T3E,” Int'l J. Applied Math. and Computational Science, vol. 11, pp.101-115, 2001.
[26] P. Ratanaworabhan, J. Ke, and M. Burtscher, “Fast Lossless Compression of Scientific Floating-Point Data,” Proc. Data Compression Conf. (DCC '06), pp. 133-142, Mar. 2006.
[27] Y. Sazeides and J.E. Smith, “The Predictability of Data Values,” Proc. 30th Int'l Symp. Microarchitecture (MICRO '97), pp. 248-258, Dec. 1997.
[28] M. Schindler, “A Fast Renormalisation for Arithmetic Coding,” Proc. Data Compression Conf. (DCC '98), p. 572, Mar. 1998.
[29] C. Touma and C. Gotsman, “Triangle Mesh Compression,” Graphics Interface, pp. 26-34, 1998.
[30] A. Trott, R. Moorhead, and J. McGenley, “Wavelets Applied to Lossless Compression and Progressive Transmission of Floating Point Data in 3-D Curvilinear Grids,” Proc. IEEE Visualization, pp.355-388, Oct. 1996.
[31] B.E. Usevitch, “JPEG2000 Extensions for Bit Plane Coding of Floating Point Data,” Proc. Data Compression Conf. (DCC '03), pp.451-461, Mar. 2003.
[32] J. Ziv and A. Lempel, “A Universal Algorithm for Data Compression,” IEEE Trans. Information Theory, vol. 23, no. 3, pp.337-343, May 1977.
17 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool