Issue No. 09 - September (2009 vol. 21)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TKDE.2008.200
Ping Luo , Chinese Academy of Sciences and Hewlett-Packard Labs China, Beijing
Hui Xiong , Rutgers University, Newark
Guoxing Zhan , Wayne State University, Detroit
Junjie Wu , Beihang University, Beijing
Zhongzhi Shi , Chinese Academy of Sciences, Beijing
This paper studies the generalization and normalization issues of information-theoretic distance measures for clustering validation. Along this line, we first introduce a uniform representation of distance measures, defined as quasi-distance, which is induced based on a general form of conditional entropy. The quasi-distance possesses three properties: symmetry, the triangle law, and the minimum reachable. These properties ensure that the quasi-distance naturally lends itself as the external measure for clustering validation. In addition, we observe that the ranges of the distance measures are different when they apply for clustering validation on different data sets. Therefore, when comparing the performances of clustering algorithms on different data sets, distance normalization is required to equalize ranges of the distance measures. A critical challenge for distance normalization is to obtain the ranges of a distance measure when a data set is provided. To that end, we theoretically analyze the computation of the maximum value of a distance measure for a data set. Finally, we compare the performances of the partition clustering algorithm K-means on various real-world data sets. The experiments show that the normalized distance measures have better performance than the original distance measures when comparing clusterings of different data sets. Also, the normalized Shannon distance has the best performance among four distance measures under study.
Clustering validation, entropy, information-theoretic distance measures, K-means clustering.
Ping Luo, Hui Xiong, Guoxing Zhan, Junjie Wu, Zhongzhi Shi, "Information-Theoretic Distance Measures for Clustering Validation: Generalization and Normalization", IEEE Transactions on Knowledge & Data Engineering, vol. 21, no. , pp. 1249-1262, September 2009, doi:10.1109/TKDE.2008.200