This Article 
 Bibliographic References 
 Add to: 
Efficient and Effective Duplicate Detection in Hierarchical Data
May 2013 (vol. 25 no. 5)
pp. 1028-1041
L. Leitão, Inst. Super. Tecnico, Porto Salvo, Portugal
P. Calado, Inst. Super. Tecnico, Porto Salvo, Portugal
M. Herschel, Lab. de Rech. en Inf. (LRI), Univ. Paris-Sud 11, Orsay, France
Although there is a long line of work on identifying duplicates in relational data, only a few solutions focus on duplicate detection in more complex hierarchical structures, like XML data. In this paper, we present a novel method for XML duplicate detection, called XMLDup. XMLDup uses a Bayesian network to determine the probability of two XML elements being duplicates, considering not only the information within the elements, but also the way that information is structured. In addition, to improve the efficiency of the network evaluation, a novel pruning strategy, capable of significant gains over the unoptimized version of the algorithm, is presented. Through experiments, we show that our algorithm is able to achieve high precision and recall scores in several data sets. XMLDup is also able to outperform another state-of-the-art duplicate detection solution, both in terms of efficiency and of effectiveness.
Index Terms:
XML,Bayesian methods,Databases,Electronic mail,Random variables,Semantics,optimization,Duplicate detection,record linkage,entity resolution,XML,Bayesian networks,data cleaning
L. Leitão, P. Calado, M. Herschel, "Efficient and Effective Duplicate Detection in Hierarchical Data," IEEE Transactions on Knowledge and Data Engineering, vol. 25, no. 5, pp. 1028-1041, May 2013, doi:10.1109/TKDE.2012.60
Usage of this product signifies your acceptance of the Terms of Use.