The Community for Technology Leaders
Green Image
Issue No. 12 - Dec. (2014 vol. 26)
ISSN: 1041-4347
pp: 2983-2996
Wei Lu , School of Computing, National University of Singapore, Singapore
Xiaoyong Du , Key Laboratory of Data Engineering and Knowledge Engineering, Ministry of Education, China
Marios Hadjieleftheriou , AT&T Labs-Research, 180 Park Ave Bldg, 103 Florham Park,
Beng Chin Ooi , School of Computing, National University of Singapore, Singapore
ABSTRACT
Edit distance is widely used for measuring the similarity between two strings. As a primitive operation, edit distance based string similarity search is to find strings in a collection that are similar to a given query string using edit distance. Existing approaches for answering such string similarity queries follow the filter-and-verify framework by using various indexes. Typically, most approaches assume that indexes and data sets are maintained in main memory. To overcome this limitation, in this paper, we propose B $^+$-tree based approaches to answer edit distance based string similarity queries, and hence, our approaches can be easily integrated into existing RDBMSs. In general, we answer string similarity search using pruning techniques employed in the metric space in that edit distance is a metric. First, we split the string collection into partitions according to a set of reference strings. Then, we index strings in all partitions using a single B$^+$ -tree based on the distances of these strings to their corresponding reference strings. Finally, we propose two approaches to efficiently answer range and KNN queries, respectively, based on the B$^+$ -tree. We prove that the optimal partitioning of the data set is an NP-hard problem, and therefore propose a heuristic approach for selecting the reference strings greedily and present an optimal partition assignment strategy to minimize the expected number of strings that need to be verified during the query evaluation. Through extensive experiments over a variety of real data sets, we demonstrate that our B$^+$ -tree based approaches provide superior performance over state-of-the-art techniques on both range and KNN queries in most cases.
INDEX TERMS
Indexing, Partitioning algorithms, Educational institutions, NP-hard problem, Electronic mail,B$^+$ -tree, Similarity search, string, edit distance
CITATION
Wei Lu, Xiaoyong Du, Marios Hadjieleftheriou, Beng Chin Ooi, "Efficiently Supporting Edit Distance Based String Similarity Search Using B $^+$-Trees", IEEE Transactions on Knowledge & Data Engineering, vol. 26, no. , pp. 2983-2996, Dec. 2014, doi:10.1109/TKDE.2014.2309131
229 ms
(Ver 3.3 (11022016))