Data Compression Conference (1996)
Mar. 31, 1996 to Apr. 3, 1996
B.J. Frey , Dept. of Electr. Eng., Toronto Univ., Ont., Canada
G.E. Hinton , Dept. of Electr. Eng., Toronto Univ., Ont., Canada
We introduce a new approach to the problem of optimal compression when a source code produces multiple codewords for a given symbol. It may seem that the most sensible codeword to use in this case is the shortest one. However, in the proposed free energy approach, random codeword selection yields an effective codeword length that can be less than the shortest codeword length. If the random choices are Boltzmann distributed, the effective length is optimal for the given source code. The expectation-maximization parameter estimation algorithms minimize this effective codeword length. We illustrate the performance of free energy coding on a simple problem where a compression factor of two is gained by using the new method.
encoding; data compression; parameter estimation; optimisation; free energy coding; optimal compression; source code; multiple codewords; random codeword selection; codeword length; Boltzmann distribution; expectation-maximization parameter estimation; compression factor
B. Frey and G. Hinton, "Free energy coding," Data Compression Conference(DCC), Snowbird, UT, 1996, pp. 73.