Best lossless compression results of color map images have been obtained by dividing the color maps into layers, and by compressing the binary layers separately by using an optimized context tree model that exploits inter-layer dependencies. In this paper, we extend the previous context tree based method to operate on color values instead of the binary layers. We generate an n-ary context tree by constructing a complete tree up to a predefined depth, and then prune out nodes that do not provide improvement in compression to generate sub-optimal context tree with incomplete structure. Experiments show that the proposed method outperforms existing methods for a large set of different color map images.