The Community for Technology Leaders
Asian Language Processing, International Conference on (2009)
Singapore
Dec. 7, 2009 to Dec. 9, 2009
ISBN: 978-0-7695-3904-1
pp: 53-58
ABSTRACT
In phrase-based statistical machine translation, the knowledge about phrase translation and phrase reordering is learned from the bilingual corpora. However, words may be poorly aligned in long sentence pairs in practice, which will then do harm to the following steps of the translation, such as phrase extraction, etc. A possible solution to this problem is segmenting long sentence pairs into shorter ones. In this paper, we present an effective approach to segmenting sentences based on the modified IBM Translation Model 1. We find that by taking into account the semantics of some words, as well as the length ratio of source and target sentences, the segmentation result is largely improved. We also discuss the effect of length factor to the segmentation result. Experiments show that our approach can improve the BLEU score of a phrase-based translation system by about 0.5 points.
INDEX TERMS
segmentation, long sentences, semantics guided, Poisson distributed length ratio, length normalization
CITATION
Biping Meng, Jiajun Chen, Xinyu Dai, Shujian Huang, "Segmenting Long Sentence Pairs for Statistical Machine Translation", Asian Language Processing, International Conference on, vol. 00, no. , pp. 53-58, 2009, doi:10.1109/IALP.2009.20
88 ms
(Ver 3.3 (11022016))