CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2009 vol.31 Issue No.08  August
Subscribe
Issue No.08  August (2009 vol.31)
pp: 13471361
Steve R. Gunn , University of Southampton, Southampton
John ShaweTaylor , University College London, London
ABSTRACT
The presence of irrelevant features in training data is a significant obstacle for many machine learning tasks. One approach to this problem is to extract appropriate features and, often, one selects a feature extraction method based on the inference algorithm. Here, we formalize a general framework for feature extraction, based on Partial Least Squares, in which one can select a userdefined criterion to compute projection directions. The framework draws together a number of existing results and provides additional insights into several popular feature extraction methods. Two new sparse kernel feature extraction methods are derived under the framework, called Sparse Maximal Alignment (SMA) and Sparse Maximal Covariance (SMC), respectively. Key advantages of these approaches include simple implementation and a training time which scales linearly in the number of examples. Furthermore, one can project a new test example using only k kernel evaluations, where k is the output dimensionality. Computational results on several realworld data sets show that SMA and SMC extract features which are as predictive as those found using other popular feature extraction methods. Additionally, on large text retrieval and face detection data sets, they produce features which match the performance of the original ones in conjunction with a Support Vector Machine.
INDEX TERMS
Machine learning, kernel methods, feature extraction, partial least squares (PLS).
CITATION
Steve R. Gunn, John ShaweTaylor, "Efficient Sparse Kernel Feature Extraction Based on Partial Least Squares", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 8, pp. 13471361, August 2009, doi:10.1109/TPAMI.2008.171
REFERENCES
