This Article 
 Bibliographic References 
 Add to: 
Feature Extraction Based on Decision Boundaries
April 1993 (vol. 15 no. 4)
pp. 388-400

A novel approach to feature extraction for classification based directly on the decision boundaries is proposed. It is shown how discriminantly redundant features and discriminantly informative features are related to decision boundaries. A procedure to extract discriminantly informative features based on a decision boundary is proposed. The proposed feature extraction algorithm has several desirable properties: (1) it predicts the minimum number of features necessary to achieve the same classification accuracy as in the original space for a given pattern recognition problem; and (2) it finds the necessary feature vectors. The proposed algorithm does not deteriorate under the circumstances of equal class means or equal class covariances as some previous algorithms do. Experiments show that the performance of the proposed algorithm compares favorably with those of previous algorithms.

[1] R. O. Duda and P. E. Hart,Pattern Classification and Scene Analysis. New York: Wiley, 1973.
[2] J. A. Richards,Remote Sensing Digital Image Analysis. New York: Spring-Verlag, 1986.
[3] K. Fukunaga and W. L. G. Koontz, "Application of the Karhunen-Loeve expansion to feature selection and ordering,"IEEE Trans. Comput., vol. C-19, no. 4, pp. 311-318, Apr. 1970.
[4] D. H. Foley and J. W. Sammon, "An optimal set of discriminant vectors,"IEEE Trans. Comput., vol. C-24, no. 3, pp. 281-289, Mar. 1975.
[5] D. Kazakos, "On the optimal linear feature,"IEEE Trans. Inform. Theory, vol. IT-24, no. 5, pp. 651-652, Sept. 1978.
[6] R. P. Heydorn, "Redundancy in feature extraction,"IEEE Trans. Comput., pp. 1051-1054, Sept. 1971.
[7] P. H. Swain and R. C. King, "Two effective feature selection criteria for multispectral remote sensing," inProc. First Int. Joint Conf. Patt. Recogn., Nov. 1973, pp. 536-540.
[8] P. Devijver and J. Kittler,Pattern Recognition: A Statistical Approach. Englewood Cliffs, NJ: Prentice-Hall, 1982.
[9] W. Malina, "On an extended Fisher criterion for feature selection,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-3, no. 5, pp. 611-614, Sept. 1981.
[10] I. D. Longstaff, "On extensions to Fisher's linear discriminant function,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-9, no. 2, pp. 321-325, Mar. 1987.
[11] S. D. Morgera and L. Datta, "Toward a fundamental theory of optimal feature selection: Part I,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-6, no. 5, pp. 601-616, Sept. 1984.
[12] K. Fukunaga,Introduction to Statistical Pattern Recognition. New York: Academic, 1972.
[13] C. G. Cullen,Matrices and Linear Transformation. Reading, MA: Addison Wesley, 1972.
[14] L. L. Biehl,et al., "A crops and soils data base for scene radiation research," inProc. Machine Processing Remotely Sensed Data Symp.(West Lafayette, IN), 1982, pp. 169-177.
[15] P. H. Swain and S. M. Davis,Remote Sensing: The Qunntitative Approach. New York: McGraw-Hill, 1978.
[16] C. Lee and D. A. Landgrebe, "Decision boundary feature selection for nonparametric classifiers," inProc. SPSE's 44th Ann. Conf.1991, pp. 475-478.
[17] C. Lee and D. A. Landgrebe, "Feature extraction and classification algorithms for high dimensional data," Ph.D dissertation, Sch. of Elect. Eng., Purdue Univ., West Lafayette, IN, 1992.

Index Terms:
Bayes methods; decision boundaries; feature extraction; classification; discriminantly redundant features; discriminantly informative features; pattern recognition; Bayes methods; decision theory; feature extraction
C. Lee, D.A. Landgrebe, "Feature Extraction Based on Decision Boundaries," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 4, pp. 388-400, April 1993, doi:10.1109/34.206958
Usage of this product signifies your acceptance of the Terms of Use.