著者
Yui Oka Katsuhito Sudoh Satoshi Nakamura
出版者
The Association for Natural Language Processing
雑誌
自然言語処理 (ISSN:13407619)
巻号頁・発行日
vol.28, no.3, pp.778-801, 2021 (Released:2021-09-15)
参考文献数
25

Neural machine translation often suffers from an under-translation problem owing to its limited modeling of the output sequence lengths. In this study, we propose a novel approach to training a Transformer model using length constraints based on length-aware positional encoding (PE). Because length constraints with exact target sentence lengths degrade the translation performance, we add a random perturbation with a uniform distribution within a certain range to the length constraints in the PE during the training. In the inference step, we predicted the output lengths from the input sequences using a length prediction model based on a large-scale pre-trained language model. In Japanese-to-English and English-to-Japanese translation, experimental results show that the proposed perturbation injection improves the robustness of the length prediction errors, particularly within a certain range.