The Community for Technology Leaders
Green Image
Issue No. 05 - May (2013 vol. 25)
ISSN: 1041-4347
pp: 1111-1124
David Marmaros , Google, Inc., Mountain View
Steven Euijong Whang , Google, Inc., Mountain View
Hector Garcia-Molina , Stanford University, Stanford
ABSTRACT
Entity resolution (ER) is the problem of identifying which records in a database refer to the same entity. In practice, many applications need to resolve large data sets efficiently, but do not require the ER result to be exact. For example, people data from the web may simply be too large to completely resolve with a reasonable amount of work. As another example, real-time applications may not be able to tolerate any ER processing that takes longer than a certain amount of time. This paper investigates how we can maximize the progress of ER with a limited amount of work using “hints,” which give information on records that are likely to refer to the same real-world entity. A hint can be represented in various formats (e.g., a grouping of records based on their likelihood of matching), and ER can use this information as a guideline for which records to compare first. We introduce a family of techniques for constructing hints efficiently and techniques for using the hints to maximize the number of matching records identified using a limited amount of work. Using real data sets, we illustrate the potential gains of our pay-as-you-go approach compared to running ER without using hints.
INDEX TERMS
Erbium, Approximation algorithms, Partitioning algorithms, Clustering algorithms, Tin, Data structures, Companies, data cleaning, Entity resolution, pay-as-you-go
CITATION
David Marmaros, Steven Euijong Whang, Hector Garcia-Molina, "Pay-As-You-Go Entity Resolution", IEEE Transactions on Knowledge & Data Engineering, vol. 25, no. , pp. 1111-1124, May 2013, doi:10.1109/TKDE.2012.43
327 ms
(Ver )