The Community for Technology Leaders
2013 IEEE 29th International Conference on Data Engineering (ICDE) (2000)
San Diego, California
Feb. 28, 2000 to Mar. 3, 2000
ISSN: 1063-6382
ISBN: 0-7695-0506-6
pp: 577
Stefan Berchtold , Software Technologie Beratung GMBH
Jörg Sander , University of Munich
Christian Böhm , Software Technologie Beratung GMBH and University of Munich
Hans-Peter Kriegel , University of Munich
H.V. Jagadish , University of Michigan
Two major approaches have been proposed to efficiently process queries in databases: Speeding up the search by using index structures, and speeding up the search by operating on a compressed database, such as a signature file. Both approaches have their limitations: Indexing techniques are inefficient in extreme configurations, such as high-dimensional spaces, where even a simple scan may be cheaper than an index-based search. Compression techniques are not very efficient in all other situations. We propose to combine both techniques to search for nearest neighbors in a high-dimensional space.For this purpose, we develop a compressed index, called the IQ-tree, with a three-level structure: The first level is a regular (flat) directory consisting of minimum bounding boxes, the second level contains data points in a compressed representation, and the third level contains the actual data.We overcome several engineering challenges in constructing an effective index structure of this type. The most significant of these is to decide how much to compress at the second level. Too much compression will lead to many needless expensive accesses to the third level. Too little compression will increase both the storage and the access cost for the first two levels.We develop a cost model and an optimization algorithm based on this cost model that permits an independent determination of the degree of compression for each second level page to minimize expected query cost. In an experimental evaluation, we demonstrate that the IQ-tree shows a performance that is the "best of both worlds" for a wide range of data distributions and dimensionalities.
Stefan Berchtold, Jörg Sander, Christian Böhm, Hans-Peter Kriegel, H.V. Jagadish, "Independent Quantization: An Index Compression Technique for High-Dimensional Data Spaces", 2013 IEEE 29th International Conference on Data Engineering (ICDE), vol. 00, no. , pp. 577, 2000, doi:10.1109/ICDE.2000.839456
107 ms
(Ver )