This Article 
 Bibliographic References 
 Add to: 
Design and Implementation of a Lossless Parallel High-Speed Data Compression System
June 2004 (vol. 15 no. 6)
pp. 481-490

Abstract—Logic density increases have made feasible the implementation of multiprocessor systems able to meet the intensive data processing demands of highly concurrent systems. This paper describes the research and hardware implementation of a high-performance parallel multicompressor chip. A detailed investigation into the performances of alternative input and output routing strategies for realistic data sets demonstrate that the design of parallel compression devices involves important trade offs that affect compression performance, latency, and throughput. The most promising approach is implemented into FPGA hardware and is shown to provide a scalable compression solution at throughputs able to cope with the demands of modern high-bandwidth applications.

[1] S. Dorward and S. Quinlan, Robust Data Compression of Network Packets Bell Labs, Lucent Technologies, networkcomp.pdf, July 2000.
[2] Recommendation V.42 bis, Data Compression Procedures for Data Circuit Terminating Equipment (DCE) Using Error Correction Procedures, CCITT (ITU), Jan. 1990.
[3] J. Ziv and A. Lempel, "A Universal Algorithm for Sequential Data Compression," IEEE Trans. Information Theory, vol. 23, no. 3, pp. 337-343, 1977.
[4] G.V. Cormack and R.N. Horspool, Data Compression Using Dynamic Markov Modelling Computer J., vol. 30, no. 6, pp. 541-550, 1987.
[5] J. Rissanen and G.G. Langdon, Arithmetic Coding IBM J. Research Development, vol. 23, no. 2, pp. 149-162, Mar. 1979.
[6] Hifn, LZS Data Compression, , July 2003.
[7] AHA, Advanced Hardware Architectures, Adaptive Lossless Data Compression,http://www.aha.comtech.php, July 2003.
[8] J.L. Núñez, Gbit/Second Lossless Data Compression Hardware doctoral thesis, Electronic and Electrical Eng., Loughborough Univ., Loughborough, 2001.
[9] J.L. Núñez and S. Jones, The X-MatchPRO 100 Mbytes/Second FPGA-Based Lossless Data Compressor Proc. Design, Automation and Test in Europe, DATE Conf., pp. 139-142, Mar. 2000.
[10] N. Ranganathan and S. Henriques, “High-Speed VLSI Designs for Lempel-Ziv-Based Data Compression,” IEEE Trans. Circuits and Systems II: Analog and Digital Signal Processing, vol. 40, pp. 96-106, Feb. 1993.
[11] B. Jung and W. Burleson, “A VLSI Systolic Array Architecture for Lempel-Ziv-Based Data Compression,” Proc. IEEE Int'l Symp. Circuits and Systems (ISCAS), pp. 65-68, May 1994.
[12] B. Jung and W.P. Burleson, Real Time VLSI Compression for High Speed Wireless Local Networks Proc. Data Compression Conf., Mar. 1995.
[13] J.A. Storer and J.H. Rief, A Parallel Architecture for High Speed Data Compression J. Parallel and Distributed Computing, vol. 13, pp. 222-227, 1991.
[14] C.-Y. Lee and R.-Y. Yang, “High-Throughput Data Compressor Designs Using Content Addressable Memory,” IEE Proc.-Circuits Devices Systems, vol. 142, pp. 69-73, Feb. 1995.
[15] B.W.Y. Wei, R. Tarver, J.-S. Kim, and K. Ng, “A Single Chip Lempel-Ziv Data Compressor,” Proc. IEEE Int'l Symp. Circuits and Systems (ISCAS), pp. 1,953-1,955, May 1993.
[16] S. Jones, Partial-Matching Lossless Data Compression Hardware IEE Proc. Conf. Computer and Digital Techniques, vol. 147, no. 5, pp. 329-334, Sept. 2000.
[17] J. Jiang and S. Jones, Parallel Design of Arithmetic Coding IEE Proc. Conf. Computer and Digital Techniques, vol. 141, no. 6, pp. 327-323, Nov. 1994.
[18] R. Stefo, J.L. Núñez, C. Feregrino, S. Mahapatra, and S. Jones, FPGA-Based Modelling Unit for High Speed Lossless Arithmetic Coding Proc. 11th Int'l Conf. Field-Programmable Logic and Applications, pp. 643-647, Aug. 2001.
[19] P.A. Franaszek, P. Heidelbeger, D.E. Poff, and J.T. Robinson, Algorithm and Data Structures for Compressed Memory Machines IBM J. Research and Development, vol. 45, no. 2, Mar. 2001.
[20] W.T. Penzhorn, A Parallel Algorithm for High-Speed Data Compression Proc. IEEE South African Symp. Comm. and Signal Processing, pp. 173-175, Sept. 1992.
[21] J.L. Simpson and C.L. Sabharwal, A Multiple Processor Approach to Data Compression Applied Computing Machinery, pp. 641-649, 1998.
[22] M.E. Gonzalez Smith and J.A. Storer, Parallel Algorithms for Data Compression J. ACM, vol. 32, no. 2, pp. 344-373, Apr. 1985.
[23] J.L. Núñez, C. Feregrino, S. Jones, and S. Bateman, X-MatchPRO: A ProASIC-Based 200 Mbytes/s Full-Duplex Lossless Data Compressor Proc. 11th Int'l Conf. Field-Programmable Logic and Applications, pp. 613-617, Aug. 2001.
[24] SystemC, System Design Language,http:/, July 2003.
[25] Handel-C, System Design Language,http:/, July 2003.
[26] R. Arnold and T. Bell, A Corpus for the Evaluation of Lossless Compression Algorithms Proc. Data Compression Conf., pp. 201-210, Mar. 1997.

Index Terms:
Lossless data compression, X-MatchProRli, Multiple Compressors, FPGA.
Mark Milward, Jos? Luis N?, David Mulvaney, "Design and Implementation of a Lossless Parallel High-Speed Data Compression System," IEEE Transactions on Parallel and Distributed Systems, vol. 15, no. 6, pp. 481-490, June 2004, doi:10.1109/TPDS.2004.7
Usage of this product signifies your acceptance of the Terms of Use.