Feature Subset Selection and Ranking for Data Dimensionality Reduction
January 2007 (vol. 29 no. 1)
pp. 162-166
DOI Bookmark:
http://doi.ieeecomputersociety.org/10.1109/TPAMI.2007.11
A new unsupervised forward orthogonal search (FOS) algorithm is introduced for feature selection and ranking. In the new algorithm, features are selected in a stepwise way, one at a time, by estimating the capability of each specified candidate feature subset to represent the overall features in the measurement space. A squared correlation function is employed as the criterion to measure the dependency between features and this makes the new algorithm easy to implement. The forward orthogonalization strategy, which combines good effectiveness with high efficiency, enables the new algorithm to produce efficient feature subsets with a clear physical interpretation.
[1] M.A. Carreira-Perpinan, “Continuous Latent Variable Models for Dimensionality Reduction and Sequential Data Reconstruction,” PhD dissertation, Dept. of Computer Science, Univ. of Sheffield, Sheffield, U.K., 2001.[2] I.K. Fodor, “A Survey of Dimension Reduction Techniques,” Technical Report UCRL-ID-148494, Lawrence Livermore Nat'l Laboratory, Center for Applied Scientific Computing, June 2002.[3] A.K. Jain, R.P.W. Duin, and J. Mao, “Statistical Pattern Recognition: A Review,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 4-37, Jan. 2000.[4] A.R. Webb, Statistical Pattern Recognition, second ed. Wiley, 2002.[5] I.T. Jolliffe, Principal Component Analysis, second ed. Springer, 2002.[6] G.P. McCabe, “Principal Variables,” Technometrics, vol. 26, pp. 137-144, May 1984.[7] W.J. Krzanowski, “Selection of Variables to Preserve Multivariate Data Structure Using Principal Components,” Applied Statististics, vol. 36, no. 1, pp. 22-33, 1987.[8] P. Mitra, C.A. Murthy, and S.K. Pal, “Unsupervised Feature Selection Using Feature Similarity,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 301-312, Mar. 2002.[9] B. Krishnapuram, A.J. Hartemink, L. Carin, and M.A.T. Figueiredo, “A Bayesian Approach to Joint Feature Selection and Classifier Design,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1105-1111, Sept. 2004.[10] M.H.C. Law, M.A.T. Figueiredo, and A.K. Jain, “Simultaneous Feature Selection and Clustering Using Mixture Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1154-1166, Sept. 2004.[11] R. Kohavi and G.H. John, “Wrappers for Feature Subset Selection,” Artificial Intelligence, vol. 97, nos. 1-2, pp. 273-324, Dec. 1997.[12] A.J. Miller, Subset Selection in Regression. Chapman and Hall, 1990.[13] P. Pudil, J. Novovicova, and J. Kittler, “Floating Search Methods in Feature Selection,” Pattern Recognition Letters, vol. 15, no. 11, pp. 1119-1125, Nov. 1994.[14] S.K. Pal, R.K. De, and J. Basak, “Unsupervised Feature Evaluation: A Neuro-Fuzzy Approach,” IEEE Trans. Neural Networks, vol. 11, no. 2, pp.366-376, Mar. 2000.[15] K.Z. Mao, “Identifying Critical Variables of Principal Components for Unsupervised Feature Selection,” IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 35, pp. 339-344, 2005.[16] I.T. Jolliffe, “Discarding Variables in a Principal Component Analysis-I: Artificial Data,” Applied Statistics, vol. 21, no. 2, pp. 160-173, 1972.[17] L. Breiman, “Statistical Modeling: The Two Cultures,” Statistical Science, vol. 16, no. 3, pp. 199-215, Aug. 2001.[18] Y. Amit and D. Geman, “Shape Quantization and Recognition with Randomized Trees,” Neural Computation, vol. 9, no. 7, pp. 1545-1588, Oct. 1997.[19] Y. Amit, D. Geman, and K. Wilder, “Joint Induction of Shape Features and Tree Classifiers,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 11, pp. 1300-1305, Nov. 1997.[20] I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, “Gene Selection for Cancer Classification Using Support Vector Machines,” Machine Learning, vol. 46, pp. 389-422, 2002.[21] M. Korenberg, S.A. Billings, Y.P. Liu, and P.J. McIlroy, “Orthogonal Parameter Estimation Algorithm for Non-Linear Stochastic Systems,” Int'l J. Control, vol. 48, pp. 193-210, 1988.[22] S.A. Billings, S. Chen, and M.J. Korenberg, “Identification of MIMO Non-Linear Systems Suing a Forward Regression Orthogonal Estimator,” Int'l J.Control, vol. 49, pp. 2157-2189, June 1989.[23] T.M. Cover and J.M. Van Campenhout, “On the Possible Orderings in the Measurement Selection Problem,” IEEE Trans. Systems, Man, and Cybernetics, vol. 7, no. 9, pp. 657-661, Sept. 1977.[24] H.L. Wei, S.A. Billings, and J. Liu, “Term and Variable Selection for Nonlinear System Identification,” Int'l J. Control, vol. 77, no. 1, pp. 86-110, Jan. 2004.[25] J.N.R. Jeffers, “Two Case Studies in the Application of Principal Component Analysis,” Applied Statistics, vol. 16, no. 3, pp. 225-236, 1967.[26] I.T. Jolliffe, “Discarding Variables in a Principal Component Analysis. II: Real Data,” Applied Statistics, vol. 22, no. 1, pp. 21-31, 1973.[27] D.J. Newman, S. Hettich, C.L. Blake, and C.J. Merz UCI Repository of Machine Learning Databases, http://www.ics.uci.edu/~mlearnMLRepository.html , 2006.[28] , Faculty of Physics, Dept. of Informatics, Nicolaus Copernicus Univ., Torun, Poland,http://www.phys.uni.torun.pl/kmk/projects datasets.html, 2006.
Index Terms:
Dimensionality reduction, feature selection, high-dimensional data.
Citation:
Hua-Liang Wei, Stephen A. Billings, "Feature Subset Selection and Ranking for Data Dimensionality Reduction," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp. 162-166, Jan. 2007, doi:10.1109/TPAMI.2007.11
Usage of this product signifies your acceptance of the
Terms of Use.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb