While multilingual neural machine translation has achieved great success, it
suffers from the off-target issue, where the translation is in the wrong
language. This problem is more pronounced on zero-shot translation tasks. In
this work, we find that failing in encoding discriminative target language
signal will lead to off-target and a closer lexical distance (i.e.,
KL-divergence) between two languages' vocabularies is related with a higher
off-target rate. We also find that solely isolating the vocab of different
languages in the decoder can alleviate the problem. Motivated by the findings,
we propose Language Aware Vocabulary Sharing (LAVS), a simple and effective
algorithm to construct the multilingual vocabulary, that greatly alleviates the
off-target problem of the translation model by increasing the KL-divergence
between languages. We conduct experiments on a multilingual machine translation
benchmark in 11 languages. Experiments show that the off-target rate for 90
translation tasks is reduced from 29\% to 8\%, while the overall BLEU score is
improved by an average of 1.9 points without extra training cost or sacrificing
the supervised directions' performance. We release the code at
\href{https://github.com/chenllliang/Off-Target-MNMT}{this https URL}
for reproduction.

本研究提出了一种名为 Language Aware Vocabulary Sharing 的算法，该算法通过增加两种语言之间的 KL-divergence，将多语言单词表构建成一个单一的单词表，有效地减少了翻译模型的离目标问题。