Color features are among the most important features used in image database retrieval, especially in cases where no additional semantic information is available. Due to its compact representation and low complexity, direct histogram comparison is the most commonly used technique in comparing color similarity of images. However, it has many serious drawbacks, including a high degree of dependency on color codebook design, sensitivity to quantization boundaries, and inefficiency in representing images with few dominant colors. In this paper, we present a new algorithm for color matching. We describe a statistical technique to extract perceptually relevant colors. We propose a new color distance measure that guaranties optimality in matching different color components of two images. Finally, experimental results are presented comparing this new algorithm to some existing techniques.