The Community for Technology Leaders
2016 IEEE 32nd International Conference on Data Engineering (ICDE) (2016)
Helsinki, Finland
May 16, 2016 to May 20, 2016
ISBN: 978-1-5090-2020-1
pp: 241-252
Antoine Boutet , University of Lyon, LIRIS, CNRS, INSA-Lyon, UMR5205, F-69621, France
Anne-Marie Kermarrec , INRIA, Rennes, France
Nupur Mittal , INRIA, Rennes, France
Francois Taiani , University of Rennes 1, France
ABSTRACT
K-Nearest-Neighbor (KNN) graphs have emerged as a fundamental building block of many on-line services providing recommendation, similarity search and classification. Constructing a KNN graph rapidly and accurately is, however, a computationally intensive task. As data volumes keep growing, speed and the ability to scale out are becoming critical factors when deploying a KNN algorithm. In this work, we present KIFF, a generic, fast and scalable KNN graph construction algorithm. KIFF directly exploits the bipartite nature of most datasets to which KNN algorithms are applied. This simple but powerful strategy drastically limits the computational cost required to rapidly converge to an accurate KNN solution, especially for sparse datasets. Our evaluation on a representative range of datasets show that KIFF provides, on average, a speed-up factor of 14 against recent state-of-the art solutions while improving the quality of the KNN approximation by 18%.
INDEX TERMS
Measurement, IP networks, Bipartite graph, Art, Motion pictures, Artificial neural networks, Search problems
CITATION
Antoine Boutet, Anne-Marie Kermarrec, Nupur Mittal, Francois Taiani, "Being prepared in a sparse world: The case of KNN graph construction", 2016 IEEE 32nd International Conference on Data Engineering (ICDE), vol. 00, no. , pp. 241-252, 2016, doi:10.1109/ICDE.2016.7498244
193 ms
(Ver 3.3 (11022016))