20th International Conference on Data Engineering (ICDE'04)
LDC: Enabling Search By Partial Distance In A Hyper-Dimensional Space
Boston, Massachusetts
March 30-April 02
ISBN: 0-7695-2065-0
Recent advances in research fields like multimedia and bioinformatics have brought about a new generation of hyper-dimensional databases which can contain hundreds or even thousands of dimensions. Such hyper-dimensional databases pose significant problems to existing high-dimensional indexing techniques which have been developed for indexing databases with (commonly) less than a hundred dimensions. To support efficient querying and retrieval on hyper-dimensional databases, we propose a methodology called Local Digital Coding (LDC) which can support k-nearest neighbors (KNN) queries on hyper-dimensional databases and yet co-exist with ubiquitous indices, such as B+-trees. LDC extracts a simple bitmap representation called Digital Code(DC) for each point in the database.Pruning during KNN search is performed by dynamically selecting only a subset of the bits from the DC based on which subsequent comparisons are performed. In doing so, expensive operations involved in computing L-norm distance functions between hyper-dimensional data can be avoided. Extensive experiments are conducted to show that our methodology offers significant performance advantages over other existing indexing methods on both real life and synthetic hyper-dimensional datasets.
Citation:
Nick Koudas, Beng Chin Ooi, Heng Tao Shen, Anthony K. H. Tung, "LDC: Enabling Search By Partial Distance In A Hyper-Dimensional Space," icde, pp.6, 20th International Conference on Data Engineering (ICDE'04), 2004