22nd International Conference on Data Engineering (ICDE'06)
A Primitive Operator for Similarity Joins in Data Cleaning
Atlanta, Georgia
April 03-April 07
ISBN: 0-7695-2570-9
Data cleaning based on similarities involves identification of "close" tuples, where closeness is evaluated using a variety of similarity functions chosen to suit the domain and application. Current approaches for efficiently implementing such similarity joins are tightly tied to the chosen similarity function. In this paper, we propose a new primitive operator which can be used as a foundation to implement similarity joins according to a variety of popular string similarity functions, and notions of similarity which go beyond textual similarity. We then propose efficient implementations for this operator. In an experimental evaluation using real datasets, we show that the implementation of similarity joins using our operator is comparable to, and often substantially better than, previous customized implementations for particular similarity functions.
Citation:
Surajit Chaudhuri, Venkatesh Ganti, Raghav Kaushik, "A Primitive Operator for Similarity Joins in Data Cleaning," icde, pp.5, 22nd International Conference on Data Engineering (ICDE'06), 2006