Neural machine translation (NMT) models are typically trained with fixed-size
input and output vocabularies, which creates an important bottleneck on their
accuracy and generalization capability. As a solution, various studies proposed
segmenting words into sub-word units and performing translation at the
sub-lexical level. However, statistical word segmentation methods have recently
shown to be prone to morphological errors, which can lead to inaccurate
translations. In this paper, we propose to overcome this problem by replacing
the source-language embedding layer of NMT with a bi-directional recurrent
neural network that generates compositional representations of the input at any
desired level of granularity. We test our approach in a low-resource setting
with five languages from different morphological typologies, and under
different composition assumptions. By training NMT to compose word
representations from character n-grams, our approach consistently outperforms
(from 1.71 to 2.48 BLEU points) NMT learning embeddings of statistically
generated sub-word units.

本文提出用双向循环神经网络替代 NMT 源语言嵌入层，在任何所需的粒度水平上生成组合表示，从而解决基于统计的分词方法中可能导致的形态错误，实现了从字符 N-gram 组成词表示的 NMT 训练方法的一致逆袭。