The Community for Technology Leaders
Green Image
Issue No. 01 - January/February (2012 vol. 9)
ISSN: 1545-5963
pp: 137-149
S. Kuruppu , Dept. of Comput. Sci. & Software Eng., Univ. of Melbourne, Parkville, VIC, Australia
Bryan Beresford-Smith , Nat. ICT Australia, Univ. of Melbourne, Parkville, VIC, Australia
T. Conway , Nat. ICT Australia, Univ. of Melbourne, Parkville, VIC, Australia
J. Zobel , Dept. of Comput. Sci. & Software Eng., Univ. of Melbourne, Parkville, VIC, Australia
ABSTRACT
Genomic repositories increasingly include individual as well as reference sequences, which tend to share long identical and near-identical strings of nucleotides. However, the sequential processing used by most compression algorithms, and the volumes of data involved, mean that these long-range repetitions are not detected. An order-insensitive, disk-based dictionary construction method can detect this repeated content and use it to compress collections of sequences. We explore a dictionary construction method that improves repeat identification in large DNA data sets. Our adaptation, Comrad, of an existing disk-based method identifies exact repeated content in collections of sequences with similarities within and across the set of input sequences. Comrad compresses the data over multiple passes, which is an expensive process, but allows Comrad to compress large data sets within reasonable time and space. Comrad allows for random access to individual sequences and subsequences without decompressing the whole data set. Comrad has no competitor in terms of the size of data sets that it can compress (extending to many hundreds of gigabytes) and, even for smaller data sets, the results are competitive compared to alternatives; as an example, 39 S. cerevisiae genomes compressed to 0.25 bits per base.
INDEX TERMS
Genomics, DNA, Approximation algorithms, Dictionaries, Algorithm design and analysis, Humans, Encoding,large data sets., Dictionary construction, compression, DNA
CITATION
S. Kuruppu, Bryan Beresford-Smith, T. Conway, J. Zobel, "Iterative Dictionary Construction for Compression of Large DNA Data Sets", IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 9, no. , pp. 137-149, January/February 2012, doi:10.1109/TCBB.2011.82
196 ms
(Ver )