Subscribe
Issue No.12 - December (2010 vol.32)
pp: 2297-2304
Sotirios P. Chatzis , Imperial College London, London
ABSTRACT
Hidden Markov models (HMMs) are a popular approach for modeling sequential data comprising continuous attributes. In such applications, the observation emission densities of the HMM hidden states are typically modeled by means of elliptically contoured distributions, usually multivariate Gaussian or Student's-t densities. However, elliptically contoured distributions cannot sufficiently model heavy-tailed or skewed populations which are typical in many fields, such as the financial and the communication signal processing domain. Employing finite mixtures of such elliptically contoured distributions to model the HMM state densities is a common approach for the amelioration of these issues. Nevertheless, the nature of the modeled data often requires postulation of a large number of mixture components for each HMM state, which might have a negative effect on both model efficiency and the training data set's size required to avoid overfitting. To resolve these issues, in this paper, we advocate for the utilization of a nonelliptically contoured distribution, the multivariate normal inverse Gaussian (MNIG) distribution, for modeling the observation densities of HMMs. As we experimentally demonstrate, our selection allows for more effective modeling of skewed and heavy-tailed populations in a simple and computationally efficient manner.
INDEX TERMS
Hidden Markov models, multivariate normal inverse Gaussian (MNIG) distribution, expectation-maximization, sequential data modeling.
CITATION
Sotirios P. Chatzis, "Hidden Markov Models with Nonelliptically Contoured State Densities", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 12, pp. 2297-2304, December 2010, doi:10.1109/TPAMI.2010.153
REFERENCES
 [1] O. Cappé, E. Moulines, and T. Rydén, Inference in Hidden Markov Models. Springer, 2005. [2] A. Dempster, N. Laird, and D. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Soc., B, vol. 39, no. 1, pp. 1-38, 1977. [3] L. Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition," Proc. IEEE, vol. 77 no. 2, pp. 245-255, Feb. 1989. [4] D. Titterington, U. Makov, and A. Smith, Statistical Analysis of Finite Mixture Distributions. Wiley, 1985. [5] G. McLachlan and D. Peel, Finite Mixture Models. Wiley, 2000. [6] S.P. Chatzis, D.I. Kosmopoulos, and T.A. Varvarigou, "Robust Sequential Data Modeling Using an Outlier Tolerant Hidden Markov Model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 9, pp. 1657-1669, Sept. 2009. [7] A. Azzalini and A. Dalla Valle, "The Multivariate Skew-Normal Distribution," Biometrika, vol. 83, no. 4, pp. 715-726, 1996. [8] A. Gupta, "Multivariate Skew $t$ -Distribution," Statistics, vol. 37, no. 4, pp. 359-363, 2003. [9] T.I. Lin, "Maximum Likelihood Estimation for Multivariate Skew Normal Mixture Models," J. Multivariate Analysis, vol. 100, pp. 257-265, 2009. [10] T. Lin, J. Lee, S. Yen, and Y. Shu, "Finite Mixture Modelling Using the Skew Normal Distribution," Statistica Sinica, vol. 17, pp. 909-927, 2007. [11] T. Zhoua and X. Heb, "Three-Step Estimation in Linear Mixed Models with Skew-$t$ Distributions," J. Statistical Planning and Inference, vol. 138, pp. 1542-1555, 2008. [12] T. Lin, J. Lee, and W. Hsieh, "Robust Mixture Modeling Using the Skew $t$ Distribution," Statistics and Computing, vol. 17, no. 2, pp. 81-92, 2007. [13] C. Leya and D. Paindaveineb, "On the Singularity of Multivariate Skew-Symmetric Models," J. Multivariate Analysis, to be published. [14] O. Barndorff-Nielsen, "Normal Inverse Gaussian Distributions and Stochastic Volatility Modelling," Scandinavian J. Statistics, vol. 24, no. 1, pp. 1-13, 1997. [15] D. Karlis, "An EM Type Algorithm for Maximum Likelihood Estimation of the Normal-Inverse Gaussian Distribution," Statistics and Probability Letters, vol. 57, no. 1, pp. 43-52, 2002. [16] T.A. Øigard, A. Hanssenb, R.E. Hansen, and F. Godtliebsen, "EM-Estimation and Modeling of Heavy-Tailed Processes with the Multivariate Normal Inverse Gaussian Distribution," Signal Processing, vol. 85, pp. 1655-1673, 2005. [17] D. Karlis and A. Santourian, "Model-Based Clustering with Non-Elliptically Contoured Distributions," Statistics and Computing, vol. 19, pp. 78-83, 2009. [18] A. Salberg, A. Swami, T. Øigard, and A. Hanssen, "The Normal Inverse Gaussian Distribution as a Model for Mui," Proc. 35th Asilomar Conf. Signals, Systems, and Computers, vol. 1, pp. 143-147, 2001. [19] M. Tweedie, "Statistical Properties of Inverse Gaussian Distributions I," Annals of Math. Statistics, vol. 28, pp. 362-377, 1957. [20] M. Tweedie, "Statistical Properties of Inverse Gaussian Distributions II," Annals of Math. Statistics, vol. 28, pp. 696-705, 1957. [21] O. Barndorff-Nielsen, J. Kent, and M. Sørensen, "Normal Variance-Mean Mixtures and z Distributions," Int'l Statistical Rev., vol. 50, no. 2, pp. 145-159, 1982. [22] O. Barndorff-Nielsen and K. Prause, "Apparent Scaling," Finance and Stochastics, vol. 5, no. 1, pp. 103-113, 2001. [23] M. Bilodeau and D. Brenner, Theory of Multivariate Statistics. Springer, 1999. [24] O. Barndorff-Nielsen, "Exponentially Decreasing Distributions for the Logarithm of Particle Size," Proc. Royal Soc. of London A, vol. 353, pp. 401-419, 1977. [25] R. Protassov, "EM-Based Maximum Likelihood Parameter Estimation for Multivariate Generalized Hyperbolic Distributions with Fixed $\lambda$ ," Statistics and Computing, vol. 14, no. 1, pp. 67-77, 2004. [26] M. Kudo, J. Toyama, and M. Shimbo, "Multidimensional Curve Classification Using Passing-Through Regions," Pattern Recognition Letters, vol. 20, nos. 11-13, pp. 1103-1111, 1999. [27] A. Asuncion and D. Newman, "UCI Machine Learning Repository," http://www.ics.uci.edu~mlearnMLRepository.html , 2007. [28] D. Kosmopoulos and I. Maglogiannis, "Hand Tracking for Gesture Recognition Tasks Using Dynamic Bayesian Network," Int'l J. Intelligent Systems and Applications, vol. 1, nos. 3/4, pp. 359-375, 2006. [29] R. Mukundan and K.R. Ramakrishnan, Moment Functions in Image Analysis: Theory and Applications. World Scientific, 1998. [30] K.F. Lee and H.W. Hon, "Speaker-Independent Phone Recognition Using Hidden Markov Models," IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 37, no. 11, pp. 1641-1648, Nov. 1989. [31] F. Sha and L.K. Saul, "Large Margin Hidden Markov Models for Automatic Speech Recognition," Advances in Neural Information Processing Systems, B. Schölkopf, J. Platt, and T. Hoffman, eds., vol. 19, pp. 1249-1256, MIT Press, 2007. [32] M.A. Bartsch and G.H. Wakefield, "Audio Thumbnailing of Popular Music Using Chroma-Based Representations," IEEE Trans. Multimedia, vol. 7, no. 1, pp. 96-104, Feb. 2005. [33] T. Giannakopoulos, D. Kosmopoulos, A. Aristidou, and S. Theodoridis, "Violence Content Classification Using Audio Features," Advances in Artificial Intelligence, pp. 502-507, Springer, 2006.