This Article 
 Bibliographic References 
 Add to: 
Semisupervised Learning of Hidden Markov Models via a Homotopy Method
February 2009 (vol. 31 no. 2)
pp. 275-287
Shihao Ji, Duke University, Durham
Layne T. Watson, Virginia Polytechnic Institute and State Univeristy, Blacksburg
Lawrence Carin, Duke University, Durham
Hidden Markov model (HMM) classifier design is considered for the analysis of sequential data, incorporating both labeled and unlabeled data for training; the balance between the use of labeled and unlabeled data is controlled by an allocation parameter \lambda \in [0, 1), where \lambda = 0 corresponds to purely supervised HMM learning (based only on the labeled data) and \lambda = 1 corresponds to unsupervised HMM-based clustering (based only on the unlabeled data). The associated estimation problem can typically be reduced to solving a set of fixed-point equations in the form of a “natural-parameter homotopy.

[1] S.J. Raudys and A.K. Jain, “Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 3, pp.252-264, Mar. 1991.
[2] X. Zhu, Z. Ghahramani, and J. Lafferty, “Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions,” Proc. 20th Int'l Conf. Machine Learning, pp. 912-919, 2003.
[3] M. Seeger, “Learning with Labeled and Unlabeled Data,” technical report, Inst. for Adaptive and Neural Computation, Univ. of Edinburgh, 2001.
[4] B. Shahshahani and D. Landgrebe, “The Effect of Unlabeled Samples in Reducing the Small Sample Size Problem and Mitigating the Hughes Phenomenon,” IEEE Trans. Geoscience and Remote Sensing, vol. 32, 1994.
[5] K. Nigam, A. Mccallum, S. Thrun, and T. Mitchell, “Text Classification from Labeled and Unlabeled Documents Using EM,” Machine Learning, vol. 39, pp. 135-167, 2000.
[6] F.G. Cozman, I. Cohen, and M.C. Cirelo, “Semi-Supervised Learning of Mixture Models,” Proc. 20th Int'l Conf. Machine Learning, 2003.
[7] M. Inoue and N. Ueda, “Exploitation of Unlabeled Sequences in Hidden Markov Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, Dec. 2003.
[8] L.R. Rabiner, “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,” Proc. IEEE, vol. 77, no. 2, pp.257-286, 1989.
[9] P. Runkle, P.K. Bharadwaj, L. Couchman, and L. Carin, “Hidden Markov Models for Multi-Aspect Target Classification,” IEEE Trans. Signal Processing, vol. 47, pp. 2035-2040, July 1999.
[10] E. Birney, “Hidden Markov Models in Biological Sequence Analysis,” IBM J. Research and Development, vol. 45, 2001.
[11] D. Cutting, J. Kupiec, J. Pedersen, and P. Sibun, “A Practical Part-of-Speech Tagger,” Proc. Third Conf. Applied Natural Language Processing, pp. 133-140, 1992.
[12] D.M. Bikel, R. Schwartz, and R.M. Weischedel, “An Algorithm that Learns What's in a Name,” Machine Learning, vol. 34, pp. 211-231, Feb. 1999.
[13] A. Dempster, N. Laird, and D. Rubin, “Maximum Likelihood from Incomplete Data via the EM Algorithm,” J. Royal Statistical Soc., vol. 39, pp. 1-38, 1977.
[14] A. Corduneanu and T. Jaakkola, “Continuation Methods for Mixing Heterogeneous Sources,” Proc. 18th Ann. Conf. Uncertainty in Artificial Intelligence, 2002.
[15] S.N. Chow, J. Mallet-Paret, and J.A. Yorke, “Finding Zeros of Maps: Homotopy Methods that Are Constructive with Probability One,” Math. of Computation, vol. 32, pp. 887-899, 1978.
[16] L.T. Watson, M. Sosonkina, R.C. Melville, A.P. Morgan, and H.F. Walker, “Algorithm 777: HOMPACK90: A Suite of Fortran 90 Codes for Globally Convergent Homotopy Algorithms,” ACM Trans. Math. Software, vol. 23, pp. 514-549, 1997.
[17] E.L. Allgower and K. Georg, Numerical Continuation Methods: An Introduction. Springer, 1990.
[18] T.M. Cover and J.A. Thomas, Elements of Information Theory. John Wiley & Sons, 1991.
[19] S. Verdú and H.V. Poor, “On Minimax Robustness: A General Approach and Applications,” IEEE Trans. Information Theory, vol. 30, no. 2, pp. 328-340, Mar. 1984.
[20] R.M. Gray, “Vector Quantization,” IEEE ASSP Magazine, pp. 4-29, Apr. 1984.

Index Terms:
Semisupervised learning, homotopy method, hidden Markov models (HMMs), supervised learning.
Shihao Ji, Layne T. Watson, Lawrence Carin, "Semisupervised Learning of Hidden Markov Models via a Homotopy Method," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 275-287, Feb. 2009, doi:10.1109/TPAMI.2008.71
Usage of this product signifies your acceptance of the Terms of Use.