Issue No.01 - Jan. (2013 vol.25)
pp: 62-75
Andrew Skabar , La Trobe University, Victoria
Khaled Abdalgader , La Trobe University, Victoria
In comparison with hard clustering methods, in which a pattern belongs to a single cluster, fuzzy clustering algorithms allow patterns to belong to all clusters with differing degrees of membership. This is important in domains such as sentence clustering, since a sentence is likely to be related to more than one theme or topic present within a document or set of documents. However, because most sentence similarity measures do not represent sentences in a common metric space, conventional fuzzy clustering approaches based on prototypes or mixtures of Gaussians are generally not applicable to sentence clustering. This paper presents a novel fuzzy clustering algorithm that operates on relational input data; i.e., data in the form of a square matrix of pairwise similarities between data objects. The algorithm uses a graph representation of the data, and operates in an Expectation-Maximization framework in which the graph centrality of an object in the graph is interpreted as a likelihood. Results of applying the algorithm to sentence clustering tasks demonstrate that the algorithm is capable of identifying overlapping clusters of semantically related sentences, and that it is therefore of potential use in a variety of text mining tasks. We also include results of applying the algorithm to benchmark data sets in several other domains.
Clustering algorithms, Prototypes, Convergence, Extraterrestrial measurements, Data models, Partitioning algorithms, graph centrality, Fuzzy relational clustering, natural language processing
Andrew Skabar, Khaled Abdalgader, "Clustering Sentence-Level Text Using a Novel Fuzzy Relational Clustering Algorithm", IEEE Transactions on Knowledge & Data Engineering, vol.25, no. 1, pp. 62-75, Jan. 2013, doi:10.1109/TKDE.2011.205
[1] V. Hatzivassiloglou, J.L. Klavans, M.L. Holcombe, R. Barzilay, M. Kan, and K.R. McKeown, "SIMFINDER: A Flexible Clustering Tool for Summarization," Proc. NAACL Workshop Automatic Summarization, pp. 41-49, 2001.
[2] H. Zha, "Generic Summarization and Keyphrase Extraction Using Mutual Reinforcement Principle and Sentence Clustering," Proc. 25th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 113-120, 2002.
[3] D.R. Radev, H. Jing, M. Stys, and D. Tam, "Centroid-Based Summarization of Multiple Documents," Information Processing and Management: An Int'l J., vol. 40, pp. 919-938, 2004.
[4] R.M. Aliguyev, "A New Sentence Similarity Measure and Sentence Based Extractive Technique for Automatic Text Summarization," Expert Systems with Applications, vol. 36, pp. 7764-7772, 2009.
[5] R. Kosala and H. Blockeel, "Web Mining Research: A Survey," ACM SIGKDD Explorations Newsletter, vol. 2, no. 1, pp. 1-15, 2000.
[6] G. Salton, Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley, 1989.
[7] J.B MacQueen, "Some Methods for Classification and Analysis of Multivariate Observations," Proc. Fifth Berkeley Symp. Math. Statistics and Probability, pp. 281-297, 1967.
[8] G. Ball and D. Hall, "A Clustering Technique for Summarizing Multivariate Data," Behavioural Science, vol. 12, pp. 153-155, 1967.
[9] J.C. Dunn, "A Fuzzy Relative of the ISODATA Process and its Use in Detecting Compact Well-Separated Clusters," J. Cybernetics, vol. 3, no. 3, pp. 32-57, 1973.
[10] J.C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, 1981.
[11] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. John Wiley & Sons, 2001.
[12] U.V. Luxburg, "A Tutorial on Spectral Clustering," Statistics and Computing, vol. 17, no. 4, pp. 395-416, 2007.
[13] B.J. Frey and D. Dueck, "Clustering by Passing Messages between Data Points," Science, vol. 315, pp. 972-976, 2007.
[14] S. Theodoridis and K. Koutroumbas, Pattern Recognition, fourth ed. Academic Press, 2008.
[15] C.D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge Univ. Press, 2008.
[16] Y. Li, D. McLean, Z.A. Bandar, J.D. O'Shea, and K. Crockett, "Sentence Similarity Based on Semantic Nets and Corpus Statistics," IEEE Trans. Knowledge and Data Eng., vol. 8, no. 8, pp. 1138-1150, Aug. 2006.
[17] R. Mihalcea, C. Corley, and C. Strapparava, "Corpus-Based and Knowledge-Based Measures of Text Semantic Similarity," Proc. 21st Nat'l Conf. Artificial Intelligence, pp. 775-780, 2006.
[18] D. Wang, T. Li, S. Zhu, and C. Ding, "Multi-Document Summarization via Sentence-Level Semantic Analysis and Symmetric Matrix Factorization," Proc. 31st Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 307-314, 2008.
[19] C. Fellbaum, WordNet: An Electronic Lexical Database. MIT Press, 1998.
[20] E.H. Ruspini, "A New Approach to Clustering," Information and Control, vol. 15, pp. 22-32, 1969.
[21] E.H. Ruspini, "Numerical Methods for Fuzzy Clustering," Information Science, vol. 2, pp. 319-350, 1970.
[22] M. Roubens, "Pattern Classification Problems and Fuzzy Sets," Fuzzy Sets and Systems, vol. 1, pp. 239-253, 1978.
[23] M.P. Windham, "Numerical Classification of Proximity Data with Assignment Measures," J. Classification, vol. 2, pp. 157-172, 1985.
[24] M.-S. Yang, "A Survey of Fuzzy Clustering," Math. Computer Modelling, vol. 18, no. 11, pp 1-16, 1993.
[25] R.J. Hathaway, J.W. Devenport, and J.C. Bezdek, "Relational Dual of the C-Means Clustering Algorithms," Pattern Recognition, vol. 22, no. 2, pp. 205-212, 1989.
[26] R.J. Hathaway and J.C. Bezdek, "NERF C-Means: Non-Euclidean Relational Fuzzy Clustering," Pattern Recognition, vol. 27, pp. 429-437, 1994.
[27] P. Corsini, F. Lazzerini, and F. Marcelloni, "A New Fuzzy Relational Clustering Algorithm Based on the Fuzzy C-Means Algorithm," Soft Computing, vol. 9, pp. 439-447, 2005.
[28] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2001.
[29] L. Kaufman and P.J. Rousseeuw, "Clustering by Means of Medoids," Statistical Analysis Based on the ${\rm L}_1$ Norm, Y. Godge, eds., pp. 405-416, North Holland/Elsevier, 1987.
[30] L. Kaufman and P.J. Rousseeuw, Finding Groups in Data. Wiley, 1990.
[31] R. Krishnapuram, A. Joshi, and Y. Liyu, "A Fuzzy Relative of the k-Medoids Algorithm with Application to Web Document and Snippet Clustering," Proc. IEEE Fuzzy Systems Conf., pp. 1281-1286, 1999.
[32] T. Geweniger, D. Zühlke, B. Hammer, and T. Villmann, "Median Fuzzy C-Means for Clustering Dissimilarity Data," Neurocomputing, vol. 73, nos. 7-9, pp. 1109-1116, 2010.
[33] T. Geweniger, D. Zühlke, B. Hammer, and T. Villmann, "Fuzzy Variant of Affinity Propagation in Comparison to Median Fuzzy c-Means," Proc. Seventh Int'l Workshop Advances in Self-Organizing Maps, pp. 72-79, 2009.
[34] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[35] M. Meila and J. Shi, "Learning Segmentation by Random Walks," Advances in Neural Information Processing Systems, vol. 14, pp. 833-840, 2001.
[36] A.Y. Ng, M.I. Jordan, and Y. Weiss, "On Spectral Clustering: Analysis and an Algorithm," Proc. Advances in Neural Information Processing Systems, pp. 849-856, 2001.
[37] S.X. Yu and J. Shi, "Multiclass Spectral Clustering," Proc. IEEE Ninth Int'l Conf. Computer Vision, pp. 11-17, 2003.
[38] D. Lee and H. Seung, "Algorithms for Non-Negative Matrix Factorization," Advances in Neural Information Processing Systems, vol. 13, pp. 556-562, 2001.
[39] S. Brin and L. Page, "The Anatomy of a Large-Scale Hypertextual Web Search Engine," Computer Networks and ISDN Systems, vol. 30, pp. 107-117, 1998.
[40] A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. the Royal Statistical Soc. Series B (Methodological), vol. 39, no. 1, pp. 1-38, 1977.
[41] U. Brandes and T. Erlebach, Network Analysis: Methodological Foundations. Springer, 2005.
[42] R. Mihalcea and P. Tarau, "TextRank: Bringing Order into Texts," Proc. Conf. Empirical Methods in Natural Language (EMNLP), pp. 404-411, 2004.
[43] G. Erkan and D.R. Radev, "LexRank: Graph-Based Lexical Centrality as Salience in Text Summarization," J. Artificial Intelligence Research, vol. 22, pp. 457-479, 2004.
[44] http:/, 2012.
[45] J.J. Jiang and D.W. Conrath, "Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy," Proc. 10th Int'l Conf. Research in Computational Linguistics, pp. 19-33, 1997.
[46] A. Budanitsky and G. Hirst, "Evaluating WordNet-Based Measures of Lexical Semantic Relatedness," Computational Linguistics, vol. 32, no. 1, pp. 13-47, 2006.
[47] J.C. Bezdek, "Cluster Validity with Fuzzy Sets," J. Cybernetics, vol. 3, no. 3, pp. 58-72, 1974.
[48] J.C. Bezdek, "Mathematical Models for Systematics and Taxonomy," Proc. Eighth Int'l Conf. Numerical Taxonomy, pp. 143-166, 1975.
[49] A. Rosenberg and J. Hirschberg, "V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure," Proc Conf. Empirical Methods in Natural Language Processing (EMNLP '07), pp. 410-420, 2007.
[50] W.M. Rand, "Objective Criteria for the Evaluation of Clustering Methods," Am. Statistical Assoc. J., vol. 66, no. 338, pp. 846-850, 1971.
[51] S. Philips, J. Pitton, and L. Atlas, "Perceptual Feature Identification for Active Sonar Echoes," Proc. IEEE OCEANS Conf., 2006.
[52] T. Hofmann and J.M. Buhmann, "Pairwise Data Clustering by Deterministic Annealing," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 1, pp. 1-14, Jan. 1997.
[53] Y. Chen, E.K. Garcia, M.R. Gupta, A. Rahimi, and L. Cazzanti, "Similarity-Based Classification: Concepts and Algorithms," J. Machine Learning Research, vol. 10, pp. 747-776, 2009.
[54] A. Asuncion and D.J. Newman, UCI Machine Learning Repository, , , Dept. of Information and Computer Science, Univ. of California, Irvine, 2012.
[55] C. Stanfill and D. Waltz, "Toward Memory-Based Reasoning," Comm. ACM, vol. 29, no. 12, pp. 1213-1228, 1986.
[56] /, 2012.
[57] reuters21578/, 2012.
[58] R.E. Bellman, Adaptive Control Processes. Princeton Univ. Press, 1961.
[59] C. Bishop, Neural Networks for Pattern Recognition. Oxford Univ. Press, 1995.