Issue No. 07 - July (2007 vol. 19)
Damien Francois , Department of Mathematical Engineering, Universite´ catholique de Louvain, Georges Lema??tre, 4, B-1348 Louvain-la-Neuve, Belgium
Vincent Wertz , Department of Mathematical Engineering, Universite´ catholique de Louvain, Georges Lema??tre, 4, B-1348 Louvain-la-Neuve, Belgium
Michel Verleysen , Microelectronic Laboratory, Place du Levant 3, B-1348 Louvain-la-Neuve, Belgium
Nearest neighbor search and many other numerical data analysis tools most often rely on the use of the euclidean distance. When data are high dimensional, however, the euclidean distances seem to concentrate; all distances between pairs of data elements seem to be very similar. Therefore, the relevance of the euclidean distance has been questioned in the past, and fractional norms (Minkowski-like norms with an exponent less than one) were introduced to fight the concentration phenomenon. This paper justifies the use of alternative distances to fight concentration by showing that the concentration is indeed an intrinsic property of the distances and not an artifact from a finite sample. Furthermore, an estimation of the concentration as a function of the exponent of the distance and of the distribution of the data is given. It leads to the conclusion that, contrary to what is generally admitted, fractional norms are not always less concentrated than the euclidean norm; a counterexample is given to prove this claim. Theoretical arguments are presented, which show that the concentration phenomenon can appear for real data that do not match the hypotheses of the theorems, in particular, the assumption of independent and identically distributed variables. Finally, some insights about how to choose an optimal metric are given.
Euclidean distance, Data analysis, Information retrieval, Content based retrieval, Nearest neighbor searches, Databases, Extraterrestrial measurements, Digital cameras, Books, DNA
D. Francois, V. Wertz and M. Verleysen, "The Concentration of Fractional Distances," in IEEE Transactions on Knowledge & Data Engineering, vol. 19, no. 7, pp. 873-886, .