The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2005 vol.17)
pp: 339-353
Kian-Lee Tan , IEEE Computer Society
ABSTRACT
In main memory systems, the L2 cache typically employs cache line sizes of 32-128 bytes. These values are relatively small compared to high-dimensional data, e.g., > 32D. The consequence is that existing techniques (on low-dimensional data) that minimize cache misses are no longer effective. In this paper, we present a novel index structure, called \Delta{\hbox{-}}{\rm{tree}}, to speed up the high-dimensional query in main memory environment. The \Delta{\hbox{-}}{\rm{tree}} is a multilevel structure where each level represents the data space at different dimensionalities: the number of dimensions increases toward the leaf level. The remaining dimensions are obtained using Principal Component Analysis. Each level of the tree serves to prune the search space more efficiently as the lower dimensions can reduce the distance computation and better exploit the small cache line size. Additionally, the top-down clustering scheme can capture the feature of the data set and, hence, reduces the search space. We also propose an extension, called \Delta^+{\hbox{-}}{\rm{tree}}, that globally clusters the data space and then partitions clusters into small regions. The \Delta^+{\hbox{-}}{\rm{tree}} can further reduce the computational cost and cache misses. We conducted extensive experiments to evaluate the proposed structures against existing techniques on different kinds of data sets. Our results show that the \Delta^+{\hbox{-}}{\rm{tree}} is superior in most cases.
INDEX TERMS
High-dimensional index, main memory, similarity query.
CITATION
Beng Chin Ooi, Bin Cui, Kian-Lee Tan, "Indexing High-Dimensional Data for Efficient In-Memory Similarity Search", IEEE Transactions on Knowledge & Data Engineering, vol.17, no. 3, pp. 339-353, March 2005, doi:10.1109/TKDE.2005.46
29 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool