This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
An Information-Theoretic Foundation for the Measurement of Discrimination Information
September 2010 (vol. 22 no. 9)
pp. 1262-1273
ASCII Text
x 
 
Di Cai, "An Information-Theoretic Foundation for the Measurement of Discrimination Information," IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 9, pp. 1262-1273, September, 2010.
 
 
BibTex
x
 
@article{ 10.1109/TKDE.2009.134,
author = {Di Cai},
title = {An Information-Theoretic Foundation for the Measurement of Discrimination Information},
journal ={IEEE Transactions on Knowledge and Data Engineering},
volume = {22},
number = {9},
issn = {1041-4347},
year = {2010},
pages = {1262-1273},
doi = {http://doi.ieeecomputersociety.org/10.1109/TKDE.2009.134},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Knowledge and Data Engineering
TI - An Information-Theoretic Foundation for the Measurement of Discrimination Information
IS - 9
SN - 1041-4347
SP1262
EP1273
EPD - 1262-1273
A1 - Di Cai,
PY - 2010
KW - Statistical semantic analysis
KW - measurement of discrimination information
KW - measurement of semantic relatedness
KW - informative term identification
KW - key term extraction
KW - text mining
KW - information retrieval.
VL - 22
JA - IEEE Transactions on Knowledge and Data Engineering
ER -
 
Di Cai, University of Wolverhampton, Wolverhampton
Hitherto, it has not been easy to interpret the meaning of the amount of discrimination information conveyed in a term rationally and explicitly within practical application contexts; it has not been simple to introduce the concept of the extent of semantic relatedness between two terms meaningfully and successfully into scientific discussions. This study is part of an attempt to do this. We attempt to answer two important questions: 1) What is the discrimination information conveyed by a term and how to measure it? 2) What is the relatedness between two terms and how to estimate it? We focus on the first question and present an in-depth investigation into the discrimination measures based on several information measures, which are widely used in a variety of applications. The relatedness measures are then naturally defined according to the individual discrimination measures. Some key points are made for clarifying potential problems arising from using the relatedness measures, and solutions are suggested. Two example applications in the contexts of text mining and information retrieval are provided. The aim of this study, of which this paper forms part, is to establish a unified theoretical framework, with measurement of discrimination information (MDI) at the core, for achieving effective measurement of semantic relatedness (MSR). Due to its generality, our method can be expected to be a useful tool with a wide range of application areas.

[1] C. Fellbaum, WordNet: An Electronic Lexical Database. The MIT Press, 1998.
[2] G. Miller, "WordNet: An On-Line Lexical Database," Int'l J. Lexicography, Special Issue, vol. 3, no. 4, pp. 235-244, 1990.
[3] P.D. Turney, "Similarity of Semantic Relations," Computational Linguistics, vol. 32, no. 3, pp. 379-410, 2006.
[4] I. Dagan, "Contextual Word Similarity," Handbook of Natural Language Processing, pp. 459-475, Marcel Dekker, Inc., 2000.
[5] A. Budanitsky and G. Hirst, "Evaluating WordNet-Based Measures of Lexical Semantic Relatedness," Computational Linguistics, vol. 4, no. 1, pp. 1-49, 2005.
[6] A. Budanitsky and G. Hirst, "Semantic Distance in WordNet: An Experimental, Application-Oriented Evaluation of Five Measures," Proc. Workshop WordNet and Other Lexical Resources, Second Meeting of the North Am. Chapter of the Assoc. for Computational Linguistics, pp. 29-34, 2001.
[7] D. Moldovan, A. Badulescu, M. Tatu, D. Antohe, and R. Girju, "Models for the Semantic Classification of Noun Phrases," Proc. Workshop Computational Lexical Semantics, pp. 60-67, 2004.
[8] V. Nastase and S. Szpakowicz, "Exploring Noun-Modifier Semantic Relations," Proc. Fifth Int'l Workshop Computational Semantics, pp. 285-301, 2003.
[9] P.D. Turney, M.L. Littman, J. Bigham, and V. Shnayder, "Combining Independent Modules to Solve Multiple-Choice Synonym and Analogy Problems," Proc. Int'l Conf. Recent Advances in Natural Language Processing, pp. 482-489, 2003.
[10] J. Morris and G. Hirst, "Lexical Cohesion Computed by Thesaural Relations as an Indicator of the Structure of Text," Computational Linguistics, vol. 17, no. 1, pp. 21-48, 1991.
[11] T.K. Landauer and S.T. Dumais, "A Solution to Plato's Problem: The Latent Semantic Analysis Theory of the Acquisition, Induction, and Representation of Knowledge," Psychological Rev., vol. 104, no. 2, pp. 211-240, 1997.
[12] K. Frantzi, S. Ananiadou, and H. Mima, "Automatic Recognition of Multi-Word Terms," Int'l J. Digital Libraries, vol. 3, no. 2, pp. 117-132, 2000.
[13] R. Florian and D. Yarowsky, "Modeling Consensus: Classifier Combination for Word Sense Disambiguation," Proc. Conf. Empirical Methods in Natural Language Processing, pp. 25-32, 2002.
[14] J.H. Lee, M.H. Kim, and Y.J. Lee, "Information Retrieval Based on Conceptual Distance in Is-A Hierarchies," J. Documentation, vol. 49, pp. 188-207, 1993.
[15] R. Richardson, A. Smeaton, and J. Murphy, "Using WordNet as a Knowledge Base for Measuring Semantic Similarity between Words," Proc. Artificial Intelligence and Cognitive Science (AICS) Conf., 1994.
[16] C. Corley and R. Mihalcea, "Measuring the Semantic Similarity of Texts," Proc. ACL Workshop Empirical Modeling of Semantic Equivalence and Entailment, pp. 13-18, 2005.
[17] I. Dagan, L. Lee, and F.C.N. Pereira, "Similarity-Based Models of Word Cooccurrence Probabilities," Machine Learning, special issue on natural language learning, vol. 34, nos. 1-3, pp. 43-69, 1999.
[18] G. Hirst and A. Budanitsky, "Correcting Real-Word Spelling Errors by Restoring Lexical Cohesion," Natural Language Eng., vol. 11, no. 1, pp. 87-111, 2005.
[19] L. Lee, "Measures of Distributional Similarity," Proc. 37th Ann. Meeting of the Assoc. for Computational Linguistics, pp. 25-32, 1999.
[20] I. Marx, Z. Dagan, J. Buhmann, and E. Shamir, "Coupled Clustering: A Method for Detecting Structural Correspondence," J. Machine Learning Research, vol. 3, pp. 747-780, 2002.
[21] S. Mohammad and G. Hirst, "Distributional Measures as Proxies for Semantic Relatedness," http://ftp.cs.toronto.edu/pub/ghMohammad+Hirst-2005.pdf , 2005.
[22] S. Mohammad and G. Hirst, "Distributional Measures of Concept-Distance: A Task-Oriented Evaluation," Proc. Conf. Empirical Methods in Natural Language Processing, 2006.
[23] S. Mohammad and G. Hirst, "Determining Word Sense Dominance Using a Thesaurus," Proc. 11th Conf. European Chapter of the Assoc. for Computational Linguistics, pp. 121-128, 2006.
[24] P. Pantel and D. Lin, "Discovering Word Senses from Text," Proc. ACM SIGKDD, pp. 613-619, 2002.
[25] P. Resnik, "Semantic Similarity in a Taxonomy: An Information-Based Measure and Its Application to Problems of Ambiguity in Natural Language," J. Artificial Intelligence Research, vol. 11, pp. 95-130, 1999.
[26] N. Seco, T. Veale, and J. Hayes, "An Intrinsic Information Content Metric for Semantic Similarity in WordNet," Proc. 16th European Conf. Artificial Intelligence, 2004.
[27] J. Weeds and D. Weir, "Co-Occurrence Retrieval: A Flexible Framework for Lexical Distributional Similarity," Computational Linguistics, vol. 31, no. 4, pp. 439-475, 2005.
[28] L. Han, L. Sun, G. Chen, and L. Xie, "ADSS: An Approach to Determining Semantic Similarity," Advances in Eng. Software, vol. 37, no. 2, pp. 129-132, 2006.
[29] J.J. Jiang and D.W. Conrath, "Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy," Proc. 10th Int'l Conf. Research in Computational Linguistics, pp. 19-33, 1997.
[30] V. Pekar and S. Staab, "Word Classification Based on Combined Measures of Distributional and Semantic Similarity," Proc. Research Note Sessions of the 10th Conf. European Chapter of the Assoc. for Computational Linguistics, pp. 147-150, 2003.
[31] M.A. Rodriguez and M.J. Egenhofer, "Determining Semantic Similarity among Entity Classes from Different Ontologies," IEEE Trans. Knowledge and Data Eng., vol. 15, no. 2, pp. 442-456, Feb. 2003.
[32] D. Cai and C.J. Van Rijsbergen, "Learning Semantic Relatedness from Term Discrimination Information," Expert Systems with Applications, vol. 36, no. 2, pp. 1860-1875, Mar. 2009.
[33] S. Kullback, Information Theory and Statistics. Wiley, 1959.
[34] R. Sibson, "Information Radius," Z. Wahrsch'Theorie and Verw. Geb, vol. 14, pp. 149-160, 1969.
[35] C.R. Rao, "Diversity: Its Measurement, Decomposition, Apportionment and Analysis," Sankhya: Indian J. Statistics, vol. 44, pp. 1-22, 1982.
[36] A. Rényi, "On Measures of Entropy and Information," Proc. Fourth Berkeley Symp. Math. Statistics and Probability, vol. 1, pp. 547-561, 1961.
[37] N. Jardine and R. Sibson, Mathematical Taxonomy. John Wiley & Sons, Ltd., 1971.
[38] C.E. Shannon, "A Mathematical Theory of Communication," Bell System and Technical J., vol. 27, pp. 379-423, 623-656, 1948.
[39] D. Cai, "IfD—Information for Discrimination," PhD dissertation, Univ. of Glasgow, 2004.
[40] D. Cai, "Determining Semantic Relatedness through the Measurement of Discrimination Information Using Jensen Difference," Int'l J. Intelligent Systems, vol. 24, no. 5, pp. 477-503, 2009.
[41] I.J. Good, Probability and the Weighing of Evidence. Charles Griffin, 1950.
[42] D. Cai and C.J. Van Rijsbergen, "Reconsidering the Fundamentals of Measurement of Discrimination Information," Proc. First Int'l Conf. Theory of Information Retrieval (ICTIR '07), pp. 151-158, 2007.
[43] TreeBASE, http://www.treebase.org/treebaseindex.html , 2009.
[44] S.E. Robertson and S. Walker, "Okapi/Keenbow at TREC-8," Proc. Eighth Text REtrieval Conf. (TREC-8), pp. 151-161, 1999.

Index Terms:
Statistical semantic analysis, measurement of discrimination information, measurement of semantic relatedness, informative term identification, key term extraction, text mining, information retrieval.
Citation:
Di Cai, "An Information-Theoretic Foundation for the Measurement of Discrimination Information," IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 9, pp. 1262-1273, Sept. 2010, doi:10.1109/TKDE.2009.134
Usage of this product signifies your acceptance of the Terms of Use.