15th International Conference on Pattern Recognition (ICPR'00) - Volume 2
Random Embedding Machines for Low-Complexity Pattern Recognition
Barcelona, Spain
September 03-September 08
ISBN: 0-7695-0750-6
Real classification problems involve structured data that can be essentially grouped into a relatively small number of clusters. It is shown that, under a local clustering condition, a set of points of a given class, embedded in binary space by a set of randomly parameterized surfaces, is linearly separable from other classes, with arbitrarily high probability. We call such a data set a local relative cluster. The size of the embedding set is linear in the input dimension and inversely proportional to the squared local clustering degree. A simple parameterization by embedding hyperplanes leads to the separation of multi-cluster data by a network with two internal layers. The computational complexity is linear in the number of relative clusters in the data. This represents a considerable reduction of the learning problem with respect to known techniques, resolving a long-standing question on the complexity of random embedding. Numerical tests show that the proposed method performs as well as state-of the-art methods, in a small fraction of the time.