This Article 
 Bibliographic References 
 Add to: 
Feature Selection: Evaluation, Application, and Small Sample Performance
February 1997 (vol. 19 no. 2)
pp. 153-158

Abstract—A large number of algorithms have been proposed for feature subset selection. Our experimental results show that the sequential forward floating selection (SFFS) algorithm, proposed by Pudil et al., dominates the other algorithms tested. We study the problem of choosing an optimal feature set for land use classification based on SAR satellite images using four different texture models. Pooling features derived from different texture models, followed by a feature selection results in a substantial improvement in the classification accuracy. We also illustrate the dangers of using feature selection in small sample size situations.

[1] T.M. Cover and J.M. Van Campenhout, "On the Possible Orderings in the Measurement Selection Problem," IEEE Trans. Systems, Man, and Cybernetics, vol. 7, no. 9, pp. 657-661, Sept. 1977.
[2] R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis. Wiley, 1973.
[3] F. Ferri, P. Pudil, M. Hatef, and J. Kittler, "Comparative Study of Techniques for Large Scale Feature Selection," Pattern Recognition in Practice IV, E. Gelsema and L. Kanal, eds., pp. 403-413. Elsevier Science B.V., 1994.
[4] Y. Hamamoto, S. Uchimura, Y. Matsunra, T. Kanaoka, and S. Tomita, "Evaluation of the Branch and Bound Algorithm for Feature Selection," Pattern Recognition Letters, vol. 11, pp. 453-456, July 1990.
[5] A.K. Jain and B. Chandrasekaran, "Dimensionality and Sample Size Considerations," Pattern Recognition in Practice, P.R. Krishnaiah and L.N. Kanal, eds., vol. 2, chap 39, pp. 835-855. North-Holland, 1982.
[6] A.K. Jain and A. Vailaya, "Image Retrieval Using Color and Shape," Pattern Recognition, vol. 29, no. 8, pp. 1,233-1,244, Aug. 1996.
[7] J. Kittler, "Feature Set Search Algorithms," Pattern Recognition and Signal Processing, C.H. Chen, ed., pp. 41-60. Sijthoff and Noordhoff, Alphen aan den Rijn, The Netherlands, 1978.
[8] J. Mao, K. Mohiuddin, and A.K. Jain, "Parsimonious Network Design and Feature Selection Through Node Pruning," Proc. 12th ICPR,Jerusalem, pp. 622-624, 1994.
[9] P.M. Narendra and K. Fukunaga, "A Branch and Bound Algorithm for Feature Subset Selection," IEEE Trans. Computers, vol. 26, no. 9, pp. 917-922, Sept. 1977.
[10] P. Pudil, J. Novovicova, and J. Kittler, "Floating Search Methods in Feature Selection," Pattern Recognition Letters, vol. 15, pp. 1,119-1,125, Nov. 1994.
[11] W.F. Punch, E.D. Goodman, M. Pei, L. Chia-Shun, P. Hovland, and R. Enbody, "Further Research on Feature Selection and Classification Using Genetic Algorithms," Proc. Fifth Int'l Conf. Genetic Algorithms, pp. 557-564, 1993.
[12] S.J. Raudys and A.K. Jain, "Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, pp. 252-264, 1991.
[13] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning Internal Representations by Error Propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D.E. Rumelhart and J.L. McClelland, eds., vol. 1, chap. 8, pp. 318-362. MIT Press, 1986.
[14] W. Siedlecki and J. Sklansky, "On Automatic Feature Selection," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 2, no. 2, pp. 197-220, 1988.
[15] W. Siedlecki and J. Sklansky, "A Note on Genetic Algorithms for Large-Scale Feature Selection," Pattern Recognition Letters, vol. 10, no. 335-347, Nov. 1989.
[16] A.H.S. Solberg and A.K. Jain, "A Study of the Invariance Properties of Textural Features," Proc. IGARS Conf., pp. 670-672,Florence, Italy, July 1995.
[17] G.V. Trunk, "A Problem of Dimensionality: A Simple Example," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 1, no. 3, no. 306-307, July 1979.
[18] Z. You and A.K. Jain, "Performance Evaluation of Shape Matching via Chord Length Distribution," Computer Vision, Graphics and Image Processing, vol. 28, pp. 185-198, 1984.
[19] B. Yu and B. Yuan, "A More Efficient Branch and Bound Algorithm for Feature Selection," Pattern Recognition, vol. 26, no. 6, pp. 883-889, 1993.

Index Terms:
Feature selection, curse of dimensionality, genetic algorithm, node pruning, texture models, SAR image classification.
Anil Jain, Douglas Zongker, "Feature Selection: Evaluation, Application, and Small Sample Performance," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 2, pp. 153-158, Feb. 1997, doi:10.1109/34.574797
Usage of this product signifies your acceptance of the Terms of Use.