This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Linear Feature Extraction for Multiclass Classification Problems Based on Class Mean and Covariance Discriminant Information
February 2006 (vol. 28 no. 2)
pp. 223-235
A parametric linear feature extraction method is proposed for multiclass classification. The skeleton of the proposed method consists of two types of schemes that are complementary to each other with regard to the discriminant information used. The approximate pairwise accuracy criterion (aPAC) and the common-mean feature extraction (CMFE) are chosen to exploit the discriminant information about class mean and about class covariance, respectively. Choosing aPAC rather than the linear discriminant analysis (LDA) can also resolve the problem of overemphasized large distances introduced by LDA, while maintaining other decent properties of LDA. To alleviate the suboptimum problem caused by a direct cascading of the two different types of schemes, there should be a mechanism for sorting and merging features based on their effectiveness. Usage of a sample-based classification error estimation for evaluation of effectiveness of features usually costs a lot of computational time. Therefore, we develop a fast spanning-tree-based parametric classification accuracy estimator as an intermediary for the aPAC and CMFE combination. The entire framework is parametric-based. This avoids paying a costly price in computation, which normally happens to the sample-based approach. Our experiments have shown that the proposed method can achieve a satisfactory performance on real data as well as simulated data.

[1] N.R. Pal and V.K. Eluri, “Two Efficient Connectionist Schemes for Structure Preserving Dimensionality Reduction,” IEEE Trans. Neural Networks, vol. 9, no. 6, pp. 1142-1154, Nov. 1998.
[2] A. Jain and D. Zongker, “Feature Selection: Evaluation, Application, and Small Sample Performance,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 2, pp. 153-158, Feb. 1997.
[3] H.M. Lee, C.M. Chen, J.M. Chen, and Y.L. Jou, “An Efficient Fuzzy Classifier with Feature Selection Based on Fuzzy Entropy,” IEEE Trans. Systems, Man, and Cybernetics B, vol. 31, no. 3, pp. 426-432, June 2001.
[4] R. Thawonmas and S. Abe, “A Novel Approach to Feature Selection Based on Analysis of Class Regions,” IEEE Trans. Systems, Man, and Cybernetics B, vol. 29, pp. 1196-1207, Apr. 1997.
[5] R.A. Fisher, “The Use of Multiple Measurements in Taxonomic Problems,” Annals of Eugenics, vol. 7, pp. 179-188, 1936.
[6] C.R. Rao, “The Utilization of Multiple Measurements in Problems of Biological Classification,” J. Royal Statistical Soc., Series B, vol. 10, pp. 159-203, 1948.
[7] K. Fukunaga, Introduction to Statistical Pattern Recognition, second ed. San Diego, Calif.: Academic Press, 1990.
[8] P.F. Hsieh and D.A. Landgrebe, “Linear Feature Extraction for Multiclass Problems,” Proc. IEEE Int'l Geoscience and Remote Sensing Symp., vol. 4, pp. 2050-2052, July 1998.
[9] M. Loog, R.P.W. Duin, and R. Haeb-Umbach, “Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 7, pp. 762-766, July 2001.
[10] C. Lee and D.A. Landgrebe, “Feature Extraction Based on Decision Boundaries,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 4, pp. 388-400, Apr. 1993.
[11] B.-C. Kuo and D.A. Landgrebe, “A Robust Classification Procedure Based on Mixture Classifiers and Nonparametric Weighted Feature Extraction,” IEEE Trans. Geoscience and Remote Sensing, vol. 40, no. 11, pp. 2486-2494, Nov. 2002.
[12] R. Lotlikar and R. Kothari, “Adaptive Linear Dimensionality Reduction for Classification,” Pattern Recognition, vol. 33, pp. 185-294, Feb. 2000.
[13] L.M. Bruce, C.H. Koger, and J. Li, “Dimensionality Reduction of Hyperspectral Data Using Discrete Wavelet Transform Feature Extraction,” IEEE Trans. Geoscience and Remote Sensing, vol. 40, no. 10, pp. 2331-2338, Oct. 2002.
[14] S. Kaewpijit, J.L. Moigne, and T.E. Ghazawi, “Automatic Reduction of Hyperspectral Imagery Using Wavelet Spectral Analysis,” IEEE Trans. Geoscience and Remote Sensing, vol. 41, no. 4, pp. 863-871, Apr. 2003.
[15] Y. Mallet, D. Coomans, J. Kautsky, and O. De Vel, “Classification Using Adaptive Wavelets for Feature Extraction,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 10, pp. 1058-1066, Oct. 1997.
[16] S. Pittner and S.V. Kamarthi, “Feature Extraction from Wavelet Coefficients for Pattern Recognition Tasks,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 1, pp. 83-88, Jan. 1999.
[17] K. Etemad and R. Chellappa, “Separability-Based Multiscale Basis Selection and Feature Extraction for Signal and Image Classification,” IEEE Trans. Image Processing, vol. 7, no. 10, pp. 1453-1465, Oct. 1998.
[18] E. Choi and C. Lee, “Optimizing Feature Extraction for Multiclass Problems,” IEEE Trans. Geoscience and Remote Sensing, vol. 39, no. 3, pp. 521-528, Mar. 2001.
[19] M. Loog, Approximate Pairwise Accuracy Criteria for Multiclass Linear Dimension Reduction: Generalizations of the Fisher Criterion, no. 44, WBBM Report Series, Delft, The Netherlands: Delft Univ. Press, 1999.
[20] E. Choi and C. Lee, “Bayes Error Evaluation of the Gaussian ML Classifier,” IEEE Trans. Geoscience and Remote Sensing, vol. 38, no. 3, pp. 1471-1475, May 2000.
[21] M.A. Carreira-Perpinan, “Mode-Finding for Mixtures of Gaussian Distributions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1318-1323, Nov. 2000.
[22] UCI Repository of Machine Learning Databases, www.ics.uci.edu/mlearnmlrepository.html, 2004.
[23] http://shay.ecn.purdue.edu/frdata/FRData/ data/78data1978ExperimentSummaries.html#781227 , LARS database, Purdue Univ., 1978.
[24] M. Loog and R.P.W. Duin, “Linear Dimensionality Reduction via a Heteroscedastic Extension of LDA: The Chernoff Criterion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 6, pp. 732-739, June 2004.

Index Terms:
Index Terms- Dimensionality reduction, linear feature extraction, discriminant analysis, classification error estimation, linear discriminant analysis, Bhattacharyya distance.
Citation:
Pi-Fuei Hsieh, Deng-Shiang Wang, Chia-Wei Hsu, "A Linear Feature Extraction for Multiclass Classification Problems Based on Class Mean and Covariance Discriminant Information," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 2, pp. 223-235, Feb. 2006, doi:10.1109/TPAMI.2006.26
Usage of this product signifies your acceptance of the Terms of Use.