Kernel methods have recently become popular for the exploration of structure in data and one of the more commonly used methods is Kernel Principal Components Analysis (Kernel PCA). This method is similar to non-linear PCA in that PCA is performed in kernel space, which is a non-linear transformation of the data into a higher dimension. We compare this method to a closely related statistic technique called Factor Analysis, show that, particularly when used in conjunction with a Varimax rotation of the factor axis, we can transform the kernel space so that the local variance in data clusters may be accounted for, and not just the global variance across all of the data clusters. When the data is matched with an appropriate kernel then this method improves the interpretability of the results.