This Article 
 Bibliographic References 
 Add to: 
Divergence Based Feature Selection for Multimodal Class Densities
February 1996 (vol. 18 no. 2)
pp. 218-223

Abstract—A new feature selection procedure based on the Kullback J-divergence between two class conditional density functions approximated by a finite mixture of parameterized densities of a special type is presented. This procedure is suitable especially for multimodal data. Apart from finding a feature subset of any cardinality without involving any search procedure, it also simultaneously yields a pseudo-Bayes decision rule. Its performance is tested on real data.

[1] D.E. Boekee and J.C.A. Van der Lubbe, "Some aspects of error bounds in feature selection," Pattern Recognition, vol. 11, pp. 353-360, 1979.
[2] A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Royal Statistical Society, vol. 39, pp. 1-38, 1977.
[3] P.A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach.Englewood Cliffs, N.J.: Prentice Hall, 1982.
[4] R. Duda and P. Hart, Pattern Classification and Scene Analysis.New York: J. Wiley, 1973.
[5] J. Grim, "On numerical evaluation of maximum-likelihood estimates for finite mixtures of distributions," Kybernetika, vol. 18, pp. 173-190, 1982.
[6] J. Grim, "Multivariate statistical pattern recognition with nonreduced dimensionality," Kybernetika, vol. 22, pp. 142-157, 1986.
[7] A.K. Jain, "Advances in statistical pattern recognition," Pattern Recognition Theory and Applications, Proc. NATO Advanced Study Inst., P. Devijver and J. Kittler, eds. Berlin-Heidelberg-New York: Springer-Verlag, pp. 1-19, 1987.
[8] P. Pudil, J. Novovicová, and J. Kittler, "Floating search methods in feature selection," Pattern Recognition Letters, vol. 15, pp. 1,119-1,125, 1994.
[9] P. Pudil, J. Novovicová, and J. Kittler, "Automatic machine learning of decision rule for classification problems in image analysis," Proc. BMVC '93—Fourth British Machine Vision Conf., vol. 1, pp. 15-24, 1993.
[10] P. Pudil, J. Novovicová, and J. Kittler, "Simultaneous learning of decision rules and important attributes for classification problems in image analysis," Image and Vision Computing, vol. 12, pp. 193-198, 1994.
[11] R.A. Redner and H.F. Walker, "Mixture densities, maximum likelihood and the EM algorithm," SIAM Review, vol. 26, pp. 195-239, 1984.

Index Terms:
Feature selection, feature ordering, mixture distribution, maximum likelihood, EM algorithm, Kullback J-divergence.
Jana Novovicová, Pavel Pudil, Josef Kittler, "Divergence Based Feature Selection for Multimodal Class Densities," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 2, pp. 218-223, Feb. 1996, doi:10.1109/34.481557
Usage of this product signifies your acceptance of the Terms of Use.