This Article 
 Bibliographic References 
 Add to: 
Gold Standard Evaluation of Ontology Learning Methods through Ontology Transformation and Alignment
November 2011 (vol. 23 no. 11)
pp. 1635-1648
Elias Zavitsanos, NCSR "Demokritos", Patriarhou Gregoriou and Neapoleos St., Athens, Athens
Georgios Paliouras, NCSR "Demokritos", Patriarhou Gregoriou and Neapoleos St., Athens, Athens
George A. Vouros, University of the Aegean, Samos Island
This paper presents a method along with a set of measures for evaluating learned ontologies against gold ontologies. The proposed method transforms the ontology concepts and their properties into a vector space representation to avoid the common string matching of concepts and properties at the lexical layer. The proposed evaluation measures exploit the vector space representation and calculate the similarity of the two ontologies (learned and gold) at the lexical and relational levels. Extensive evaluation experiments are provided, which show that these measures capture accurately the deviations from the gold ontology. The proposed method is tested using the Genia and the Lonely Planet gold ontologies, as well as the ontologies in the benchmark series of the Ontology Alignment Evaluation Initiative.

[1] R. Porzel and R. Malaka, "A Task-Based Approach for Ontology Evaluation," Proc. ECAI Workshop Ontology Learning and Population (OLP), 2004.
[2] P. Velardi, R. Navigli, A. Cuchiarelli, and F. Neri, "Evaluation of OntoLearn, a Methodology for Automatic Learning of Domain Ontologies," Ontology Learning from Text: Methods, Evaluation and Applications, IOS Press, 2005.
[3] C. Brewster, H. Alani, S. Dasmahapatra, and Y. Wilks, "Data Driven Ontology Evaluation," Proc. Int'l Conf. Language Resources and Evaluation, 2004.
[4] A.B. Jones, C.V. Storey, V. Sugumaran, and P. Ahluwalia, "A Semiotic Suite for Assessing the Quality of Ontologies," Data and Knowledge Eng., vol. 55, no. 1, pp. 84-102, 2004.
[5] A. Lozano-Tello and A. Gomez-Perez, "Ontometric: A Method to Choose the Appropriate Ontology," J. Database Management, vol. 15, no 2, pp. 1-18, 2004.
[6] A. Lozano-Tello, A. Gomez-Perez, and E. Sosa, "Selection of Ontologies for the Semantic Web," Proc. Int'l Conf. Web Eng. (ICWE '03), pp. 413-416, 2003.
[7] J. Brank, D. Mladenić, and M. Grobelnik, "Gold Standard Based Ontology Evaluation Using Instance Assignment," Proc. Fourth Workshop Evaluating Ontologies for Web EON '06, 2006.
[8] K. Dellschaft and S. Staab, "On How to Perform a Gold Standard Based Evaluation of Ontology Learning," Proc. Fifth Int'l Conf. Semantic Web, 2006.
[9] D. Maynard, W. Peters, and Y. Li, "Metrics for Evaluation of Ontology-Based Information Extraction," Proc. Workshop Evaluating Ontologies for Web EON '06, 2006.
[10] A. Meadche and S. Staab, "Measuring Similarity between Ontologies," Proc. European Conf. Knowledge Acquisition and Management (EKAW), pp. 251-263, 2002.
[11] E. Zavitsanos, G. Paliouras, and G.A. Vouros, "A Distributional Approach to Evaluating Ontology Learning Methods Using a Gold Standard," Proc. European Conf. Artificial Intelligence (ECAI '08) Ontology Learning and Population (OLP '08) Workshop, 2008.
[12] I.V. Levenshtein, "Binary Codes Capable of Correcting Deletions, Insertions and Reversals," Cybernetics and Control Theory, vol. 10, no. 8, pp. 707-710, 1966.
[13] M. Ehrig, P. Haase, N. Stohanovic, and M. Hefke, "Similarity for Ontologies—A Comprehensive Framework," Proc. European Conf. Information Systems, 2005.
[14] A. Meadche and S. Staab, "Ontology Learning for the Semantic Web," IEEE Intelligent Systems, vol. 16, no. 2, pp. 72-79, Mar./Apr. 2001.
[15] M. Sabou, C. Wroe, C. Goble, and H. Stuckenschmidt, "Learning Domain Ontologies for Semantic Web Service Descriptions," J. Web Semantics, vol. 3, no. 4, pp. 340-365, 2005.
[16] E. Zavitsanos, S. Petridis, G. Paliouras, and G.A. Vouros, "Determining Automatically the Size of Learned Ontologies," Proc. 18th European Conf. Artificial Intelligence (ECAI), 2008.
[17] H.F. Witschel, "Using Decision Trees and Text Mining Techniques for Extending Taxonomies," Proc. Workshop Learning and Extending Lexical Ontologies by Using Machine Learning Methods, 2005.
[18] A. Wagner, "Enriching a Lexical Semantic Net with Selectional Preferences by Means of Statistical Corpus Analysis," ECAI 2000: Proc. First Workshop Ontology Learning, pp. 37-42, 2000.
[19] B. Ganter and R. Wille, Formal Concept Analysis: Mathematical Foundations. Springer-Verlag, 1999.
[20] A.L. Gibbs and F.E. Su, "On Choosing and Bounding Probability Metrics," Int'l Statistical Rev., vol. 70, no. 3, pp. 419-435, 2002.
[21] Ontology Alignment Evaluation Initiative, http:/oaei., 2011.
[22] Y. Kalfoglou and M. Schorlemmer, "Ontology Mapping: The State of the Art," The Knowledge Eng. Rev., vol. 18, no. 1, pp. 1-31, 2003.
[23] P. Wang and B. Xu, "Lily: Ontology Alignment Results for OAEI 2008," Proc. Seventh Int'l Semantic Web Conf. (ISWC '08) Workshop Ontology Matching, 2008.
[24] Y.R. Jean-Mary and M.R. Kabuka, "ASMOV: Results for OAEI 2008," Proc. Int'l Semantic Web Conf. (ISWC '08) Workshop Ontology Matching, 2008.

Index Terms:
Knowledge valuation, machine learning, concept learning, ontology design.
Elias Zavitsanos, Georgios Paliouras, George A. Vouros, "Gold Standard Evaluation of Ontology Learning Methods through Ontology Transformation and Alignment," IEEE Transactions on Knowledge and Data Engineering, vol. 23, no. 11, pp. 1635-1648, Nov. 2011, doi:10.1109/TKDE.2010.195
Usage of this product signifies your acceptance of the Terms of Use.