This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Simulated Annealing Using a Reversible Jump Markov Chain Monte Carlo Algorithm for Fuzzy Clustering
April 2005 (vol. 17 no. 4)
pp. 479-490
ASCII Text
x 
 
Sanghamitra Bandyopadhyay, "Simulated Annealing Using a Reversible Jump Markov Chain Monte Carlo Algorithm for Fuzzy Clustering," IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 4, pp. 479-490, April, 2005.
 
 
BibTex
x
 
@article{ 10.1109/TKDE.2005.64,
author = {Sanghamitra Bandyopadhyay},
title = {Simulated Annealing Using a Reversible Jump Markov Chain Monte Carlo Algorithm for Fuzzy Clustering},
journal ={IEEE Transactions on Knowledge and Data Engineering},
volume = {17},
number = {4},
issn = {1041-4347},
year = {2005},
pages = {479-490},
doi = {http://doi.ieeecomputersociety.org/10.1109/TKDE.2005.64},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Knowledge and Data Engineering
TI - Simulated Annealing Using a Reversible Jump Markov Chain Monte Carlo Algorithm for Fuzzy Clustering
IS - 4
SN - 1041-4347
SP479
EP490
EPD - 479-490
A1 - Sanghamitra Bandyopadhyay,
PY - 2005
KW - Pattern recognition
KW - fuzzy clustering
KW - cluster validity index
KW - determining the number of clusters
KW - Reversible Jump Markov Chain Monte Carlo
KW - simulated annealing
KW - remote sensing.
VL - 17
JA - IEEE Transactions on Knowledge and Data Engineering
ER -
 
In this paper, an approach for automatically clustering a data set into a number of fuzzy partitions with a simulated annealing using a Reversible Jump Markov Chain Monte Carlo algorithm is proposed. This is in contrast to the widely used fuzzy clustering scheme, the Fuzzy C-Means (FCM) algorithm, which requires the a priori knowledge of the number of clusters. The said approach performs the clustering by optimizing a cluster validity index, the Xie-Beni index. It makes use of the homogeneous Reversible Jump Markov Chain Monte Carlo (RJMCMC) kernel as the proposal so that the algorithm is able to jump between different dimensions, i.e., number of clusters, until the correct value is obtained. Different moves, like birth, death, split, merge, and update, are used for sampling a candidate state given the current state. The effectiveness of the proposed technique in optimizing the Xie-Beni index and thereby determining the appropriate clustering is demonstrated for both artificial and real-life data sets. In a part of the investigation, the utility of the fuzzy clustering scheme for classifying pixels in an IRS satellite image of Kolkata is studied. A technique for reducing the computation efforts in the case of satellite image data is incorporated.

[1] A.K. Jain and R.C. Dubes, Algorithms for Clustering Data. Englewood Cliffs, N.J.: Prentice Hall, 1988.
[2] R. Krishnapuram and J. Keller, “A Possibilistic Approach to Clustering,” IEEE Trans. Fuzzy Sets, vol. 1, pp. 98-110, 1993.
[3] J.C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms. New York: Plenum, 1981.
[4] J.C. Dunn, “A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters,” J. Cybernetics, vol. 3, pp. 32-57, 1973.
[5] D.L. Davies and D.W. Bouldin, “A Cluster Separation Measure,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 1, pp. 224-227, 1979.
[6] R.E. Hammah and J.H. Curran, “Validity Measures for the Fuzzy Cluster Analysis of Orientations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1467-1472, Dec. 2000.
[7] J.C. Bezdek, “Cluster Validity with Fuzzy Sets,” J. Cybernetics, vol. 3, pp. 58-72, 1974.
[8] J.C. Bezdek, “Mathematical Models for Systematics and Taxonomy,” Proc. Eighth Int'l Conf. Numerical Taxonomy, G. Estabrook, ed., pp. 143-166, 1975.
[9] M.P. Windham, “Cluster Validity for the Fuzzy C-Means Clustering Algorithm,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 4, pp. 357-363, 1982.
[10] A.M. Bensaid, L.O. Hall, J.C. Bezdek, L.P. Clarke, M.L. Silbiger, J.A. Arrington, and R.F. Murtagh, “Validity-Guided Reclustering with Applications to Image Segmentation,” IEEE Trans. Fuzzy Systems, vol. 4, no. 2, pp. 112-123, 1996.
[11] X.L. Xie and G. Beni, “A Validity Measure for Fuzzy Clustering,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, pp. 841-847, July 1991.
[12] N.R. Pal and J.C. Bezdek, “On Cluster Validity for the Fuzzy C-Means Model,” IEEE Trans. Fuzzy Systems, vol. 3, pp. 370-379, 1995.
[13] S. Kirkpatrik, C. Gelatt, and M. Vecchi, “Optimization by Simulated Annealing,” Science, vol. 220, pp. 671-680, 1983.
[14] P.J.M. van Laarhoven and E.H.L. Aarts, Simulated Annealing: Theory and Applications. Kluwer Academic, 1987
[15] E. Forgy, “Cluster Analysis of Multivariate Data: Efficiency vs. Interpretability of Classifications,” Biometrics, vol. 21, no. 3, p. 768, 1965.
[16] S. Bandyopadhyay, U. Maulik, and M.K. Pakhira, “Clustering Using Simulated Annealing with Probabilistic Redistribution,” Int'l J. Pattern Recognition and Artificial Intelligence, vol. 15, no. 2, pp. 269-285, 2001.
[17] R.W. Klein and R.C. Dubes, “Experiments in Projection and Clustering by Simulated Annealing,” Pattern Recognition, 1989.
[18] S.Z. Selim and K. Alsultan, “A Simulated Annealing Algorithm for the Clustering Problem,” Pattern Recognition, vol. 24, pp. 1003-1008, 1991.
[19] A.V. Lukashin and R. Fuchs, “Analysis of Temporal Gene Expression Profiles: Clustering by Simulated Annealing and Determining the Optimal Number of Clusters,” Bioinformatics, vol. 17, no. 5, pp. 405-419, 2001.
[20] D.E. Brown and C.L. Huntley, “A Practical Application of Simulated Annealing to Clustering,” Pattern Recognition, vol. 25, pp. 401-412, 1992.
[21] S. Bandyopadhyay and U. Maulik, “An Evolutionary Technique Based on K-Means Algorithm for Optimal Clustering in ${\hbox{\rlap{I}\kern 2.0pt{\hbox{R}}}}^N$ ,” Information Sciences, vol. 146, pp. 221-237, 2002.
[22] K. Rose, G. Gurewitz, and G.C. Fox, “A Deterministic Annealing Approach to Clustering,” Pattern Recognition Letters, vol. 11, pp. 589-594, 1990.
[23] T. Hofmann and J. Buhmann, “Pairwise Data Clustering by Deterministic Annealing,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 19, no. 1, pp. 1-14, Jan. 1997.
[24] B. Bakker and T. Heskes, “Model Clustering by Deterministic Annealing,” Proc. European Symp. Artificial Neural Networks, pp. 87-92, 1999.
[25] L. Hermes and J.M. Buhmann, “Semi-Supervised Image Segmentation by Parametric Distributional Clustering,” Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 229-245, 2003.
[26] J.M. Garibaldi and E.C. Ifeachor, “Application of Simulated Annealing Fuzzy Model Tuning to Umbilical Cord Acid-Base Interpretation,” IEEE Trans. Fuzzy Systems, vol. 7, no. 1, pp. 72-84, 1999.
[27] O. Cordon, F. Herrera, and P. Villar, “Analysis and Guidelines to Obtain a Good Uniform Fuzzy Partition Granularity for Fuzzy Rule-Based Systems Using Simulated Annealing,” Int'l J. Approximate Reasoning, vol. 25, no. 3 pp. 187-216, 2000.
[28] P.J. Green, “Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination,” Biometrika, vol. 82, pp. 711-732, 1995.
[29] W.K. Hastings, “Monte Carlo Sampling Methods Using Markov Chains and Their Applications,” Biometrika, vol. 57, pp. 97-109, 1970.
[30] S.P. Wilson and J. Zerubia, “Segmentation of Textured Satellite and Aerial Images by Bayesian Inference and Markov Fields,” Technical Report 4336, INRIA, Sophia-Antipolis, France, 2001.
[31] A. Andrieu, J.F.G. de Freitas, and A. Doucet, “Robust Full Bayesian Learning for Radial Basis Networks,” Neural Computation, vol. 13, pp. 2359-2407, 2001.
[32] H. Akaike, “A New Look at Statistical Model Identification,” IEEE Trans. Automatic Control, vol. 19, pp. 716-723, 1974.
[33] S. Mitra and S.K. Pal, “Fuzzy Multi-Layer Perceptron, Inferencing and Rule Generation,” IEEE Trans. Neural Networks, vol. 6, pp. 51-63, 1995.
[34] U. Maulik and S. Bandyopadhyay, “Fuzzy Partitioning Using a Real-Coded Variable-Length Genetic Algorithm for Pixel Classification,” IEEE Trans. Geoscience and Remote Sensing, vol. 41, no. 5, pp. 1075-1081, 2003.
[35] E. Levine and E. Domany, “Resampling Method for Unsupervised Estimation of Cluster Validity,” Neural Computation, vol. 13, no. 11, pp. 2573-2593, 2001.

Index Terms:
Pattern recognition, fuzzy clustering, cluster validity index, determining the number of clusters, Reversible Jump Markov Chain Monte Carlo, simulated annealing, remote sensing.
Citation:
Sanghamitra Bandyopadhyay, "Simulated Annealing Using a Reversible Jump Markov Chain Monte Carlo Algorithm for Fuzzy Clustering," IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 4, pp. 479-490, April 2005, doi:10.1109/TKDE.2005.64
Usage of this product signifies your acceptance of the Terms of Use.