Issue No. 01 - January (2006 vol. 18)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TKDE.2006.11
Andrew Back , IEEE
Determining the most appropriate inputs to a model has a significant impact on the performance of the model and associated algorithms for classification, prediction, and data analysis. Previously, we proposed an algorithm ICAIVS which utilizes independent component analysis (ICA) as a preprocessing stage to overcome issues of dependencies between inputs, before the data being passed through to an inout variable selection (IVS) stage. While we demonstrated previously with artificial data that ICA can prevent an overestimation of necessary input variables, we show here that mixing between input variables is common in real-world data sets so that ICA preprocessing is useful in practice. This experimental test is based on new measures introduced in this paper. Furthermore, we extend the implementation of our variable selection scheme to a statistical dependency test based on mutual information and test several algorithms on Gaussian and sub-Gaussian signals. Specifically, we propose a novel method of quantifying linear dependencies using ICA estimates of mixing matrices with a new Linear Mixing Measure (LMM).
Index Terms- Input variable selection, modeling, data preprocessing, independent component analysis, mutual information estimation.
A. Back, T. Trappenberg and J. Ouyang, "Input Variable Selection: Mutual Information and Linear Mixing Measures," in IEEE Transactions on Knowledge & Data Engineering, vol. 18, no. , pp. 37-46, 2006.