| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
Class Conditional Density Estimation Using Mixtures with Constrained Component Sharing
July 2003 (vol. 25 no. 7)
pp. 924-928
Abstract—We propose a generative mixture model classifier that allows for the class conditional densities to be represented by mixtures having certain subsets of their components shared or common among classes. We argue that, when the total number of mixture components is kept fixed, the most efficient classification model is obtained by appropriately determining the sharing of components among class conditional densities. In order to discover such an efficient model, a training method is derived based on the EM algorithm that automatically adjusts component sharing. We provide experimental results with good classification performance.
[1] 924 C.M. Bishop, Neural Networks for Pattern Recognition. Clarendon Press, 1995.[2] L.R. Bahl, M. Padmanabhan, D. Nahamoo, and P.S. Gopalakrishnan, Discriminative Training of Gaussian Mixture Models for Large Vocabulary Speech Recognition Systems Proc. Int'l Conf. Acoustics, Speech, and Signal Processing, 1996.[3] C.L. Blake and C.J. Merz, UCI Repository of Machine Learning Databases. Dept. of Computer and Information Sciences, Univ. of California, Irvine, 1998.[4] M. Brand, An Entropic Estimator for Structure Discovery Neural Information Processing Systems 11, 1998.[5] A.P. Dempster, N.M. Laird, and D.B. Rubin, Maximum Likelihood Estimation from Incomplete Data via the EM Algorithm J. Royal Statistical Soc. B, vol. 39, pp. 1-38, 1977.[6] R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.[7] Z. Ghahramani and M.I. Jordan, Supervised Learning From Incomplete Data Via an EM Approach Neural Information Processing Systems 7, Mass.: MIT Press, D.J. Cowan, G. Tesauro, and J. Alspector, eds., vol. 6, pp. 120-127, 1994.[8] T.J. Hastie and R.J. Tibshirani, Discriminant Analysis by Gaussian Mixtures J. Royal Statistical Soc. B, vol. 58, pp. 155-176, 1996.[9] T. Jebara and A. Pentland, Maximum Conditional Likelihood Via Bound Maximization and the CEM Algorithm Neural Information Processing Systems 11, M. Kearns, S. Solla, and D. Cohn, eds., 1998.[10] M.I. Jordan and R.A. Jacobs, “Hierarchical Mixtures of Experts and the EM Algorithm,” Neural Computation, vol. 6, pp. 181-214, 1994.[11] P. Kontkanen, P. Myllymaki, and H. Tirri, Comparing Presequential Model Selection Criteria in Supervised Learning of Mixture Models Proc. Eighth Int'l Workshop Artificial Intelligence and Statistics, T. Jaakola and T. Richardson, eds., pp. 233-238, 2001.[12] G.J. McLachlan and D. Peel, Finite Mixture Models. Wiley, 2000.[13] D.J. Miller and H.S. Uyar, A Mixture of Experts Classifier With Learning Based on Both Labeled and Unlabeled Data Neural Information Processing Systems 9, Mass.: MIT Press, M.C. Mozer, M.I. Jordan, and T. Petsche, eds., 1996.[14] R. Redner and H. Walker, Mixture Densities, Maximum Likelihood and the EM Algorithm SIAM Rev., vol. 26, no. 2, pp. 195-239, 1984.[15] L.K. Saul and D.D. Lee, Multiplicative Updates for Classification by Mixture Models Neural Information Processing Systems 14, S. Becker, T. Dietterich, and Z. Ghahramani, eds., 2002.[16] M. Titsias and A. Likas, Shared Kernel Models for Class Conditional Density Estimation IEEE Trans. Neural Networks, vol. 12, no. 5, pp. 987-997, Sept. 2001.[17] D.M. Titterington, A.F. Smith, and U.E. Makov, Statistical Analysis of Finite Mixture Distributions. Wiley, 1985.
Index Terms:
Mixture models, classification, density estimation, EM algorithm, component sharing.
Citation:
Michalis K. Titsias, Aristidis Likas, "Class Conditional Density Estimation Using Mixtures with Constrained Component Sharing," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 924-928, July 2003, doi:10.1109/TPAMI.2003.1206521
