Issue No. 05 - May (2013 vol. 35)
W. K. Cheung , Dept. of Comput. Sci., Hong Kong Baptist Univ., Hong Kong, China
Jiming Liu , Dept. of Comput. Sci., Hong Kong Baptist Univ., Hong Kong, China
Jia Zeng , Sch. of Comput. Sci. & Technol., Soochow Univ., Suzhou, China
Latent Dirichlet allocation (LDA) is an important hierarchical Bayesian model for probabilistic topic modeling, which attracts worldwide interest and touches on many important applications in text mining, computer vision and computational biology. This paper represents the collapsed LDA as a factor graph, which enables the classic loopy belief propagation (BP) algorithm for approximate inference and parameter estimation. Although two commonly used approximate inference methods, such as variational Bayes (VB) and collapsed Gibbs sampling (GS), have gained great success in learning LDA, the proposed BP is competitive in both speed and accuracy, as validated by encouraging experimental results on four large-scale document datasets. Furthermore, the BP algorithm has the potential to become a generic scheme for learning variants of LDA-based topic models in the collapsed space. To this end, we show how to learn two typical variants of LDA-based topic models, such as author-topic models (ATM) and relational topic models (RTM), using BP based on the factor graph representations.
Indexes, Approximation algorithms, Hidden Markov models, Approximation methods, Joints, Inference algorithms, Computational modeling, variational Bayes, Latent Dirichlet allocation, topic models, belief propagation, message passing, factor graph, Bayesian networks, Markov random fields, hierarchical Bayesian models, Gibbs sampling
W. K. Cheung, Jiming Liu, Jia Zeng, "Learning Topic Models by Belief Propagation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 35, no. , pp. 1121-1134, May 2013, doi:10.1109/TPAMI.2012.185