The Community for Technology Leaders
RSS Icon
Issue No.09 - September (2009 vol.31)
pp: 1657-1669
Sotirios P. Chatzis , University of Miami, Coral Gables
Dimitrios I. Kosmopoulos , NSCR Demokritos, Athens
Theodora A. Varvarigou , National Technical University of Athens, Athens
Hidden Markov (chain) models using finite Gaussian mixture models as their hidden state distributions have been successfully applied in sequential data modeling and classification applications. Nevertheless, Gaussian mixture models are well known to be highly intolerant to the presence of untypical data within the fitting data sets used for their estimation. Finite Student's t-mixture models have recently emerged as a heavier-tailed, robust alternative to Gaussian mixture models, overcoming these hurdles. To exploit these merits of Student's t-mixture models in the context of a sequential data modeling setting, we introduce, in this paper, a novel hidden Markov model where the hidden state distributions are considered to be finite mixtures of multivariate Student's t-densities. We derive an algorithm for the model parameters estimation under a maximum likelihood framework, assuming full, diagonal, and factor-analyzed covariance matrices. The advantages of the proposed model over conventional approaches are experimentally demonstrated through a series of sequential data modeling applications.
Hidden Markov models, student's t-distribution, expectation-maximization, factor analysis, sequential data modeling.
Sotirios P. Chatzis, Dimitrios I. Kosmopoulos, Theodora A. Varvarigou, "Robust Sequential Data Modeling Using an Outlier Tolerant Hidden Markov Model", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 9, pp. 1657-1669, September 2009, doi:10.1109/TPAMI.2008.215
[1] O. Cappé, E. Moulines, and T. Rydén, Inference in Hidden Markov Models. 2005.
[2] L. Rabiner, “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,” Proc. IEEE, vol. 77, pp.245-255, 1989.
[3] A. Dempster, N. Laird, and D. Rubin, “Maximum Likelihood from Incomplete Data via the EM Algorithm,” J. Royal Statistical Soc. B, vol. 39, no. 1, pp.1-38, 1977.
[4] D. Peel and G. McLachlan, “Robust Mixture Modeling Using the$t$ Distribution,” Statistics and Computing, vol. 10, pp.335-344, 2000.
[5] S. Chatzis, D. Kosmopoulos, and T. Varvarigou, “Signal Modeling and Classification Using a Robust Latent Space Model Based on $t$ Distributions,” IEEE Trans. Signal Processing, vol. 56, no. 3, pp.949-963, 2008.
[6] M. Svensén and C.M. Bishop, “Robust Bayesian Mixture Modelling,” Neurocomputing, vol. 64, pp.235-252, 2005.
[7] S. Chatzis and T. Varvarigou, “Factor Analysis Latent Subspace Modeling and Robust Fuzzy Clustering Using $t$ Distributions,” IEEE Trans. Fuzzy Systems, to appear.
[8] L. Arslan and J. Hansen, “Selective Training for Hidden Markov Models with Applications to Speech Classification,” IEEE Trans. Speech and Audio Processing, vol. 7, no. 1, pp.46-54, 1999.
[9] C.-H. Lee and S.-Y. Lee, “Noise-Robust Speech Recognition Using Top-Down Selective Attention with an HMM Classifier,” IEEE Signal Processing Letters, vol. 14, no. 7, pp.489-491, 2007.
[10] J.-T. Chien and C.-P. Liao, “Maximum Confidence Hidden Markov Modeling for Face Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 4, pp.606-616, Apr. 2008.
[11] F. Sha and L.K. Saul, “Large Margin Hidden Markov Models for Automatic Speech Recognition,” Advances in Neural Information Processing Systems, vol. 19, B. Schölkopf, J. Platt, and T. Hoffman, eds., pp.1249-1256, MIT Press, 2007.
[12] C. Liu and D. Rubin, “ML Estimation of the $t$ Distribution Using EM and Its Extensions, ECM and ECME,” Statistica Sinica, vol. 5, no. 1, pp.19-39, 1995.
[13] D. Andrews and C. Mallows, “Scale Mixtures of Normal Distributions,” J. Royal Stat. Soc. B, vol. 36, pp.99-102, 1974.
[14] G. McLachlan, R. Bean, and L.B.-T. Jones, “Extension of the Mixture of Factor Analyzers Model to Incorporate the MultiVariate $t$ -Distribution,” Computational Statistics and Data Analysis, vol. 51, no. 11, pp.5327-5338, 2006.
[15] C. Archambeau and M. Verleysen, “Robust Bayesian Clustering,” Neural Networks, vol. 20, pp.129-138, 2007.
[16] S. Dharanipragada and K. Visweswariah, “Gaussian Mixture Models with Covariances or Precisions in Shared Multiple Subspaces,” IEEE Trans. Audio, Speech, and Language Processing, vol. 14, no. 4, pp.1255-1266, 2006.
[17] J. Lindblom and J. Samuelsson, “Bounded Support Gaussian Mixture Modeling of Speech Spectra,” IEEE Trans. Speech and Audio Processing, vol. 11, no. 1, pp.88-99, 2003.
[18] J. Frankel and S. King, “Speech Recognition Using Linear Dynamic Models,” IEEE Trans. Audio, Speech, and Language Processing, vol. 15, no. 1, pp.246-256, 2007.
[19] M. Tipping and C. Bishop, “Mixtures of Probabilistic Principal Component Analyzers,” Neural Computation, vol. 11, no. 2, pp.443-482, 1999.
[20] Z. Ghahramani and G. Hinton, “The EM Algorithm for Mixtures of Factor Analyzers,” Technical Report CRGTR-96-1, Dept. of Computer Science, Univ. of Toronto, 1997.
[21] X. Meng and D. van Dyk, “The EM Algorithm—An Old Folk Song Sung to a Fast New Tune (with Discussion),” J. Royal Statistical Soc. B, vol. 59, no. 3, pp.511-567, 1997.
[22] D. Kosmopoulos and I. Maglogiannis, “Hand Tracking for Gesture Recognition Tasks Using Dynamic Bayesian Network,” Int'l J. Intelligent Systems and Applications, vol. 1, nos.3/4, pp.359-375, 2006.
[23] R. Mukundan and K.R. Ramakrishnan, Moment Functions in Image Analysis: Theory and Applications. World Scientific, 1998.
[24] L.F. Lamel, R.H. Kassel, and S. Seneff, “Speech Database Development: Design and Analysis of the Acoustic-Phonetic Corpus,” Proc. DARPA Speech Recognition Workshop, pp.100-109, 1986.
[25] K.F. Lee and H.W. Hon, “Speaker-Independent Phone Recognition Using Hidden Markov Models,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 37, no. 11, pp.1641-1648, 1988.
[26] T.M. Mitchell, R. Hutchinson, R.S. Niculescu, F. Pereira, X. Wang, M. Just, and S. Newman, “Learning to Decode Cognitive States from Brain Images,” Machine Learning, vol. 57, no. 1, pp.145-175, 2004.
[27] G. McLachlan and D. Peel, Finite Mixture Models. Wiley, 2000.
17 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool