Data Compression Conference (DCC '97) Significantly Lower Entropy Estimates for Natural DNA Sequences March 25-March 27 ISBN: 0-8186-7761-9
If DNA were a random string over its alphabet {A,C,G,T}, an optimal code would assign 2 bits to each nucleotide. We imagine DNA to be a highly ordered, purposeful molecule, and might therefore reasonably expect statistical models of its string representation to produce much lower entropy estimates. Surprisingly this has not been the case for many natural DNA sequences, including portions of the human genome. We introduce a new statistical model (compression algorithm), the strongest reported to date, for naturally occurring DNA sequences. Conventional techniques code a nucleotide using only slightly fewer bits (1.90) than one obtains by relying only on the frequency statistics of individual nucleotides (1.95). Our method in some cases increases this gap by more than five-fold (1.66) and may lead to better performance in microbiological pattern recognition applications. One of our main contributions, and the principle source of these improvements, is the formal inclusion of inexact match information in the model. The existence of matches at various distances forms a panel of experts which are then combined into a single prediction. The structure of this combination is novel and its parameters are learned using expectation maximization (EM).
Index Terms:
genetics, natural DNA sequences, entropy estimates, alphabet, random string, optimal code, nucleotide, statistical models, string representation, human genome, compression algorithm, frequency statistics, microbiological pattern recognition applications, inexact match information, distances, prediction, expectation maximization, parameters
Citation:
David Loewenstern, Peter N. Yianilos, "Significantly Lower Entropy Estimates for Natural DNA Sequences," dcc, pp.151, Data Compression Conference (DCC '97), 1997 Usage of this product signifies your acceptance of the Terms of Use. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||