We study the problem of learning a non-parametric mapping between two continuous spaces without having access to input-output pairs for training, but rather to groups of input-output pairs, where the correspondence structure within each group is unknown and where outliers may be present. This problem is solved by transforming each space using the channel representation, and finding a linear mapping on the transformed domain. The asymptotical behavior of the method for a large number of training samples is found to be very related to the case of known correspondences. The results are evaluated on simulated data.