Publication 2013 Issue No. 6 - June Abstract - The Infinite-Order Conditional Random Field Model for Sequential Data Modeling
The Infinite-Order Conditional Random Field Model for Sequential Data Modeling
June 2013 (vol. 35 no. 6)
pp. 1523-1534
 ASCII Text x Sotirios P. Chatzis, Yiannis Demiris, "The Infinite-Order Conditional Random Field Model for Sequential Data Modeling," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1523-1534, June, 2013.
 BibTex x @article{ 10.1109/TPAMI.2012.208,author = {Sotirios P. Chatzis and Yiannis Demiris},title = {The Infinite-Order Conditional Random Field Model for Sequential Data Modeling},journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},volume = {35},number = {6},issn = {0162-8828},year = {2013},pages = {1523-1534},doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2012.208},publisher = {IEEE Computer Society},address = {Los Alamitos, CA, USA},}
 RefWorks Procite/RefMan/Endnote x TY - JOURJO - IEEE Transactions on Pattern Analysis and Machine IntelligenceTI - The Infinite-Order Conditional Random Field Model for Sequential Data ModelingIS - 6SN - 0162-8828SP1523EP1534EPD - 1523-1534A1 - Sotirios P. Chatzis, A1 - Yiannis Demiris, PY - 2013KW - Computational modelingKW - Data modelsKW - ContextKW - Hidden Markov modelsKW - Inference algorithmsKW - Context modelingKW - Approximation methodsKW - mean-field principleKW - Conditional random fieldKW - sequential dataKW - sequence memoizerVL - 35JA - IEEE Transactions on Pattern Analysis and Machine IntelligenceER -
Sotirios P. Chatzis, Cyprus University of Technology, Limassol
Yiannis Demiris, Imperial College London, London
Sequential data labeling is a fundamental task in machine learning applications, with speech and natural language processing, activity recognition in video sequences, and biomedical data analysis being characteristic examples, to name just a few. The conditional random field (CRF), a log-linear model representing the conditional distribution of the observation labels, is one of the most successful approaches for sequential data labeling and classification, and has lately received significant attention in machine learning as it achieves superb prediction performance in a variety of scenarios. Nevertheless, existing CRF formulations can capture only one- or few-timestep interactions and neglect higher order dependences, which are potentially useful in many real-life sequential data modeling applications. To resolve these issues, in this paper we introduce a novel CRF formulation, based on the postulation of an energy function which entails infinitely long time-dependences between the modeled data. Building blocks of our novel approach are: 1) the sequence memoizer (SM), a recently proposed nonparametric Bayesian approach for modeling label sequences with infinitely long time dependences, and 2) a mean-field-like approximation of the model marginal likelihood, which allows for the derivation of computationally efficient inference algorithms for our model. The efficacy of the so-obtained infinite-order CRF ($({\rm CRF}^{\infty })$) model is experimentally demonstrated.
Index Terms:
Computational modeling,Data models,Context,Hidden Markov models,Inference algorithms,Context modeling,Approximation methods,mean-field principle,Conditional random field,sequential data,sequence memoizer
Citation:
Sotirios P. Chatzis, Yiannis Demiris, "The Infinite-Order Conditional Random Field Model for Sequential Data Modeling," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1523-1534, June 2013, doi:10.1109/TPAMI.2012.208