Data Compression Conference (DCC '95)
Universal coding for arbitrarily varying sources and for hierarchies of model classes
Snowbird, Utah
March 28-March 30
ISBN: 0-8186-7012-6
The minimum redundancy attainable by universal lossless codes for finite-state arbitrarily varying sources (AVS), and, in general, for an hierarchy of model classes, is investigated. In the AVS case, if the space of all possible underlying state sequences is partitioned into types, then the minimum universal coding redundancy can be essentially lower bounded by a quantity that decomposes into two terms, the first of which is the minimum redundancy within the type class (i.e., intra-type class redundancy), and the second is the minimum redundancy associated with a class of sources that can be thought of as "representatives" of the different types (i.e., inter-type class redundancy). This behavior can be generalized to universal coding for hierarchy of model classes, where each model class in the hierarchy has an increasing complexity. The lower bound for coding with respect to hierarchy of models is achievable by a Shannon code w.r.t an appropriate two-stage mixture, where the first stage mixture is over the sources in each class, and the second is a mixture over the indices of the model classes.
Index Terms:
source coding; redundancy; channel capacity; entropy; universal coding; arbitrarily varying sources; minimum redundancy; universal lossless codes; finite-state sources; hierarchy of model classes; underlying state sequences; lower bound; intra-type class redundancy; inter-type class redundancy; Shannon code; two-stage mixture
Citation:
M. Feder, N. Merhav, "Universal coding for arbitrarily varying sources and for hierarchies of model classes," dcc, pp.82, Data Compression Conference (DCC '95), 1995