This Article 
 Bibliographic References 
 Add to: 
A Least Biased Fuzzy Clustering Method
September 1994 (vol. 16 no. 9)
pp. 954-960

A new operational definition of cluster is proposed, and a fuzzy clustering algorithm with minimal biases is formulated by making use of the maximum entropy principle to maximize the entropy of the centroids with respect to the data points (clustering entropy). The authors make no assumptions on the number of clusters or their initial positions. For each value of an adimensional scale parameter /spl beta/', the clustering algorithm makes each data point iterate towards one of the cluster's centroids, so that both hard and fuzzy partitions are obtained. Since the clustering algorithm can make a multiscale analysis of the given data set one can obtain both hierarchy and partitioning type clustering. The relative stability with respect to /spl beta/' of each cluster structure is defined as the measurement of cluster validity. The authors determine the specific value of /spl beta/' which corresponds to the optimal positions of cluster centroids by minimizing the entropy of the data points with respect to the centroids (clustered entropy). Examples are given to show how this least biased method succeeds in getting perceptually correct clustering results.

[1] A. K. Jain and R. C. Dubes,Algorithms for Clustering Data. Englewood Cliffs, NJ: Prentice-Hall, 1988.
[2] G. H. Ball and D. J. Hall, "Isodata, a novel method of data analysis and pattern classification," Stanford Res. Inst. Tech. Rep. (NTIS AD 699616), Stanford, CA, 1965.
[3] J. C. Bezdek, "Fuzzy mathematics in pattern classification," Ph.D. dissertation, Cornell Univ., Ithaca, NY, 1973.
[4] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, "Optimization by simulated annealing,"Science, vol. 220, pp. 671-680, 1983.
[5] S. Geman and D. Geman, "Stochastic relaxation, Gibbs distribution, and bayesian restoration of images,"IEEE Trans. Pattern. Anal. Machine Intell., vol. PAMI-6, pp. 721-741, 1984.
[6] K. Rose, E. Gurewitz, and G. Fox, "A deterministic annealing approach to clustering,"Pattern Recognit. Lett., vol. 11, no. 9, pp. 589-594, 1990.
[7] R. O. Duda and P. E. Hart,Pattern Classification and Scene Analysis. New York: Wiley, 1973.
[8] I. Gath and A. B. Geva, "Unsupervised optimal fuzzy clustering,"IEEE Trans. Pattern Anal. Machine Intell.. vol. PAMI-6, pp. 721-728, 1989.
[9] X.-L. Xie and G. Beni, "Validity measure for fuzzy clustering,"IEEE Trans. Pattern Anal. Machine Intell., vol. 13, no. 8, pp. 841-847, Aug. 1991.
[10] B. S. Everitt,Cluster Analysis. New York: John Wiley, 1974.
[11] R. Dubes and A. K. Jain, "Clustering techniques: The user's dilemma,"Pattern Recognit., vol. 8, pp. 247-260, 1976.
[12] G. Matthews and J. Hearne, "Clustering without a metric,"IEEE Trans. Pattern Anal. Machine Intell., vol. 13, no. 2, pp. 175-184, Feb. 1991.
[13] E. T. Jaynes, "Information theory and statistical mechanics," inPapers on Probability, Statistics, and Statistical Physics, R. D. Rosenkrantz, Ed. Dordrecht, The Netherlands, Kluwer Academic Publishers, 1989 (Reprint of a 1957 paper).
[14] R. A. Fisher, "Multiple measurements in taxonomic problems,"Ann. Eugenics, vol. 7, pp. 179-188, 1936.
[15] M. James,Classification Algorithm. New York: John Wiley, 1985, ch. 7.
[16] G. Beni and X. Liu, "An efficiency computation algorithm for the least biased fuzzy clustering method," in preparation.

Index Terms:
entropy; fuzzy set theory; pattern recognition; least biased fuzzy clustering method; minimal biases; maximum entropy principle; clustering entropy; hard partitions; fuzzy partitions; multiscale analysis; relative stability; cluster structure
G. Beni, X. Liu, "A Least Biased Fuzzy Clustering Method," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 9, pp. 954-960, Sept. 1994, doi:10.1109/34.310694
Usage of this product signifies your acceptance of the Terms of Use.