Issue No. 03 - March (2011 vol. 23)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TKDE.2010.134
Debabrata Dey , University of Washington, Seattle
Vijay S. Mookerjee , University of Texas at Dallas, Richardson
Dengpan Liu , University of Alabama in Huntsville, Huntsville
The need to consolidate the information contained in heterogeneous data sources has been widely documented in recent years. In order to accomplish this goal, an organization must resolve several types of heterogeneity problems, especially the entity heterogeneity problem that arises when the same real-world entity type is represented using different identifiers in different data sources. Statistical record linkage techniques could be used for resolving this problem. However, the use of such techniques for online record linkage could pose a tremendous communication bottleneck in a distributed environment (where entity heterogeneity problems are often encountered). In order to resolve this issue, we develop a matching tree, similar to a decision tree, and use it to propose techniques that reduce the communication overhead significantly, while providing matching decisions that are guaranteed to be the same as those obtained using the conventional linkage technique. These techniques have been implemented, and experiments with real-world and synthetic databases show significant reduction in communication overhead.
Record linkage, entity matching, sequential decision making, decision tree, data heterogeneity.
D. Liu, V. S. Mookerjee and D. Dey, "Efficient Techniques for Online Record Linkage," in IEEE Transactions on Knowledge & Data Engineering, vol. 23, no. , pp. 373-387, 2010.