This Article 
 Bibliographic References 
 Add to: 
Hybrid Genetic Algorithms for Feature Selection
November 2004 (vol. 26 no. 11)
pp. 1424-1437
This paper proposes a novel hybrid genetic algorithm for feature selection. Local search operations are devised and embedded in hybrid GAs to fine-tune the search. The operations are parameterized in terms of their fine-tuning power, and their effectiveness and timing requirements are analyzed and compared. The hybridization technique produces two desirable effects: a significant improvement in the final performance and the acquisition of subset-size control. The hybrid GAs showed better convergence properties compared to the classical GAs. A method of performing rigorous timing analysis was developed, in order to compare the timing requirement of the conventional and the proposed algorithms. Experiments performed with various standard data sets revealed that the proposed hybrid GA is superior to both a simple GA and sequential search algorithms.

[1] F.Z. Brill, D.E. Brown, and W.N. Martin, Fast Genetic Selection of Features for Neural Network Classifiers IEEE Trans. Neural Networks, vol. 3, no. 2, pp. 324-328, Mar. 1992.
[2] T.N. Bui and B.R. Moon, Genetic Algorithm and Graph Partitioning IEEE Trans. Computers, vol. 45, no. 7, pp. 841-855, July 1996.
[3] M. Dash and H. Liu, Feature Selection for Classification Intelligent Data Analysis, vol. 1, no. 3, pp. 131-156, 1997.
[4] L. Davis, Handbook of Genetic Algorithms. Van Nostrand Reinhold, 1991.
[5] F.J. Ferri, P. Pudil, M. Hatef, and J. Kittler, Comparative Study of Techniques for Large-Scale Feature Selection Pattern Recognition in Practice IV, E.S. Gelsema and L.N. Kanal, eds., pp. 403-413, 1994.
[6] J. Holland, Adaptation in Nature and Artificial Systems. MIT Press, 1992.
[7] C.-C. Hung, A. Fahsi, W. Tadesse, and T. Coleman, A Comparative Study of Remotely Sensed Data Classification Using Principal Components Analysis and Divergence Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, pp. 2444-2449, 1997.
[8] H. Ishikawa, Multiscale Feature Selection in Stereo Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 132-137, 1999.
[9] A. Jain and D. Zongker, Feature Selection: Evaluation, Application, and Small Sample Performance IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 2, pp. 153-158, Feb. 1997.
[10] P. Jog, J. Suh, and D. Gucht, The Effect of Population Size, Heuristic Crossover and Local Improvement on a Genetic Algorithm for the Traveling Salesman Problem Proc. Int'l Conf. Genetic Algorithms, pp. 110-115, 1989.
[11] J. Kittler, Feature Selection and Extraction Handbook of Pattern Recognition and Image Processing, T.Y. Young and K.S. Fu, eds., pp. 59-83, 1986.
[12] M. Kudo and J. Sklansky, Comparison of Algorithms that Select Features for Pattern Recognition Pattern Recognition, vol. 33, no. 1, pp. 25-41, 2000.
[13] L.I. Kuncheva and L.C. Jain, Nearest Neighbor Classifier: Simultaneous Editing and Feature Selection Pattern Recognition Letters, vol. 20, pp. 1149-1156, 1999.
[14] P. Langley, Selection of Relevant Features in Machine Learning Proc. AAAI Fall Symp. Relevance, pp. 1-5, 1994.
[15] Principles of Visual Information Retrieval. M.S. Lew, ed., Springer, 2001.
[16] Y. Liu and F. Dellaert, A Classification Based Similarity Metric for 3D Image Retrieval Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 800-805, 1998.
[17] M.J. Martin-Bautista and M.-A. Vila, A Survey of Genetic Feature Selection in Mining Issues Proc. 1999 Congress on Evolutionary Computation (CEC '99), pp. 1314-1321, July 1999.
[18] K. Messer and J. Kittler, Using Feature Selection to Aid an Iconic Search through an Image Database Proc. EEE Int'l Conf. Acoustics, Speech, and Signal processing (ICASSP), vol. 4, pp. 2605-2608, 1997.
[19] P.M. Murphy and D.W. Aha, UCI Repository for Machine Learning Databases technical report, Dept. of Information and Computer Science, Univ. of California, Irvine, Calif., 1994, .
[20] P.M. Narendra and K. Fukunaga, A Branch and Bound Algorithm for Feature Subset Selection IEEE Trans. Computers, vol. 26, no. 9, pp. 917-922, Sept. 1977.
[21] I.-S. Oh and C.Y. Suen, Distance Features for Neural Network-Based Recognition of Handwritten Characters Int'l J. Document Analysis and Recognition, vol. 1, no. 2, pp. 73-88, 1998.
[22] I.-S. Oh, J.-S. Lee, and C.Y. Suen, Analysis of Class Separation and Combination of Class-Dependent Features for Handwriting Recognition IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp. 1089-1094, Oct. 1999.
[23] I.-S. Oh, J.-S. Lee, and B.-R. Moon, Local Search-Embedded Genetic Algorithms for Feature Selection Proc. Int'l Conf. Pattern Recognition, 2002.
[24] S. Piramuthu, Evaluating Feature Selection Methods for Learning in Data Mining Applications Proc. 31st Ann. Hawaii Int'l Conf. System Science, pp. 294-301, 1998.
[25] P. Pudil, J. Novovicova, and J. Kittler, Floating Search Methods in Feature Selection Pattern Recognition Letters, vol. 15, pp. 1119-1125, 1994.
[26] S. Puuronen, A. Tsymbal, and I. Skrypnik, Advanced Local Feature Selection in Medical Diagnostics Proc. 13th IEEE Symp. Computer-Based Medical Systems, pp. 25-30, 2000.
[27] M.L. Raymer, W.F. Punch, E.D. Goodman, L.A. Kuhn, and A.K. Jain, Dimensionality Reduction Using Genetic Algorithms IEEE Trans. Evolutionary Computation, vol. 4, no. 2, pp. 164-171, July 2000.
[28] W. Siedlecki and J. Sklansky, On Automatic Feature Selection Int'l J. Pattern Recognition and Artificial Intelligence, vol. 2, no. 2, pp. 197-220, 1988.
[29] W. Siedlecki and J. Sklansky, A Note on Genetic Algorithms for Large-Scale Feature Selection Pattern Recognition Letters, vol. 10, pp. 335-347, 1989.
[30] J.H. Yang and V. Honavar, Feature Subset Selection Using a Genetic Algorithm IEEE Intelligent Systems, vol. 13, no. 2, pp. 44-49, 1998.
[31] B. Yu and B. Yuan, A More Efficient Branch and Bound Algorithm for Feature Selection Pattern Recognition, vol. 26, no. 6, pp. 883-889, 1993.
[32] X. Zheng, B.A. Julstrom, and W. Cheng, Design of Vector Quantization Codebooks Using a Genetic Algorithm Proc. IEEE Int'l Conf. Evolutionary Computation, pp. 525-529, 1997.

Index Terms:
Feature selection, hybrid genetic algorithm, sequential search algorithm, local search operation, atomic operation, multistart algorithm.
Il-Seok Oh, Jin-Seon Lee, Byung-Ro Moon, "Hybrid Genetic Algorithms for Feature Selection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 11, pp. 1424-1437, Nov. 2004, doi:10.1109/TPAMI.2004.105
Usage of this product signifies your acceptance of the Terms of Use.