The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - February (2009 vol.31)
pp: 275-287
Shihao Ji , Duke University, Durham
Layne T. Watson , Virginia Polytechnic Institute and State Univeristy, Blacksburg
Lawrence Carin , Duke University, Durham
ABSTRACT
Hidden Markov model (HMM) classifier design is considered for the analysis of sequential data, incorporating both labeled and unlabeled data for training; the balance between the use of labeled and unlabeled data is controlled by an allocation parameter \lambda \in [0, 1), where \lambda = 0 corresponds to purely supervised HMM learning (based only on the labeled data) and \lambda = 1 corresponds to unsupervised HMM-based clustering (based only on the unlabeled data). The associated estimation problem can typically be reduced to solving a set of fixed-point equations in the form of a ?natural-parameter homotopy.
INDEX TERMS
Semisupervised learning, homotopy method, hidden Markov models (HMMs), supervised learning.
CITATION
Shihao Ji, Layne T. Watson, Lawrence Carin, "Semisupervised Learning of Hidden Markov Models via a Homotopy Method", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 2, pp. 275-287, February 2009, doi:10.1109/TPAMI.2008.71
REFERENCES
[1] S.J. Raudys and A.K. Jain, “Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 3, pp.252-264, Mar. 1991.
[2] X. Zhu, Z. Ghahramani, and J. Lafferty, “Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions,” Proc. 20th Int'l Conf. Machine Learning, pp. 912-919, 2003.
[3] M. Seeger, “Learning with Labeled and Unlabeled Data,” technical report, Inst. for Adaptive and Neural Computation, Univ. of Edinburgh, 2001.
[4] B. Shahshahani and D. Landgrebe, “The Effect of Unlabeled Samples in Reducing the Small Sample Size Problem and Mitigating the Hughes Phenomenon,” IEEE Trans. Geoscience and Remote Sensing, vol. 32, 1994.
[5] K. Nigam, A. Mccallum, S. Thrun, and T. Mitchell, “Text Classification from Labeled and Unlabeled Documents Using EM,” Machine Learning, vol. 39, pp. 135-167, 2000.
[6] F.G. Cozman, I. Cohen, and M.C. Cirelo, “Semi-Supervised Learning of Mixture Models,” Proc. 20th Int'l Conf. Machine Learning, 2003.
[7] M. Inoue and N. Ueda, “Exploitation of Unlabeled Sequences in Hidden Markov Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, Dec. 2003.
[8] L.R. Rabiner, “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,” Proc. IEEE, vol. 77, no. 2, pp.257-286, 1989.
[9] P. Runkle, P.K. Bharadwaj, L. Couchman, and L. Carin, “Hidden Markov Models for Multi-Aspect Target Classification,” IEEE Trans. Signal Processing, vol. 47, pp. 2035-2040, July 1999.
[10] E. Birney, “Hidden Markov Models in Biological Sequence Analysis,” IBM J. Research and Development, vol. 45, 2001.
[11] D. Cutting, J. Kupiec, J. Pedersen, and P. Sibun, “A Practical Part-of-Speech Tagger,” Proc. Third Conf. Applied Natural Language Processing, pp. 133-140, 1992.
[12] D.M. Bikel, R. Schwartz, and R.M. Weischedel, “An Algorithm that Learns What's in a Name,” Machine Learning, vol. 34, pp. 211-231, Feb. 1999.
[13] A. Dempster, N. Laird, and D. Rubin, “Maximum Likelihood from Incomplete Data via the EM Algorithm,” J. Royal Statistical Soc., vol. 39, pp. 1-38, 1977.
[14] A. Corduneanu and T. Jaakkola, “Continuation Methods for Mixing Heterogeneous Sources,” Proc. 18th Ann. Conf. Uncertainty in Artificial Intelligence, 2002.
[15] S.N. Chow, J. Mallet-Paret, and J.A. Yorke, “Finding Zeros of Maps: Homotopy Methods that Are Constructive with Probability One,” Math. of Computation, vol. 32, pp. 887-899, 1978.
[16] L.T. Watson, M. Sosonkina, R.C. Melville, A.P. Morgan, and H.F. Walker, “Algorithm 777: HOMPACK90: A Suite of Fortran 90 Codes for Globally Convergent Homotopy Algorithms,” ACM Trans. Math. Software, vol. 23, pp. 514-549, 1997.
[17] E.L. Allgower and K. Georg, Numerical Continuation Methods: An Introduction. Springer, 1990.
[18] T.M. Cover and J.A. Thomas, Elements of Information Theory. John Wiley & Sons, 1991.
[19] S. Verdú and H.V. Poor, “On Minimax Robustness: A General Approach and Applications,” IEEE Trans. Information Theory, vol. 30, no. 2, pp. 328-340, Mar. 1984.
[20] R.M. Gray, “Vector Quantization,” IEEE ASSP Magazine, pp. 4-29, Apr. 1984.
14 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool