This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Pay-As-You-Go Entity Resolution
May 2013 (vol. 25 no. 5)
pp. 1111-1124
Steven Euijong Whang, Google, Inc., Mountain View
David Marmaros, Google, Inc., Mountain View
Hector Garcia-Molina, Stanford University, Stanford
Entity resolution (ER) is the problem of identifying which records in a database refer to the same entity. In practice, many applications need to resolve large data sets efficiently, but do not require the ER result to be exact. For example, people data from the web may simply be too large to completely resolve with a reasonable amount of work. As another example, real-time applications may not be able to tolerate any ER processing that takes longer than a certain amount of time. This paper investigates how we can maximize the progress of ER with a limited amount of work using “hints,” which give information on records that are likely to refer to the same real-world entity. A hint can be represented in various formats (e.g., a grouping of records based on their likelihood of matching), and ER can use this information as a guideline for which records to compare first. We introduce a family of techniques for constructing hints efficiently and techniques for using the hints to maximize the number of matching records identified using a limited amount of work. Using real data sets, we illustrate the potential gains of our pay-as-you-go approach compared to running ER without using hints.
Index Terms:
Erbium,Approximation algorithms,Partitioning algorithms,Clustering algorithms,Tin,Data structures,Companies,data cleaning,Entity resolution,pay-as-you-go
Citation:
Steven Euijong Whang, David Marmaros, Hector Garcia-Molina, "Pay-As-You-Go Entity Resolution," IEEE Transactions on Knowledge and Data Engineering, vol. 25, no. 5, pp. 1111-1124, May 2013, doi:10.1109/TKDE.2012.43
Usage of this product signifies your acceptance of the Terms of Use.