Recently, token-level adaptive training has achieved promising improvement in
machine translation, where the cross-entropy loss function is adjusted by
assigning different training weights to different tokens, in order to alleviate
the token imbalance problem. However, previous approaches only use static word
frequency information in the target language without considering the source
language, which is insufficient for bilingual tasks like machine translation.
In this paper, we propose a novel bilingual mutual information (BMI) based
adaptive objective, which measures the learning difficulty for each target
token from the perspective of bilingualism, and assigns an adaptive weight
accordingly to improve token-level adaptive training. This method assigns
larger training weights to tokens with higher BMI, so that easy tokens are
updated with coarse granularity while difficult tokens are updated with fine
granularity. Experimental results on WMT14 English-to-German and WMT19
Chinese-to-English demonstrate the superiority of our approach compared with
the Transformer baseline and previous token-level adaptive training approaches.
Further analyses confirm that our method can improve the lexical diversity.

本文提出了一种新的双语互信息（BMI）自适应目标函数，通过从双语视角度量每个目标单词的学习难度并相应地分配自适应权重来改进基于令牌级别的自适应训练，实验结果表明与变压器基线和以前的令牌级别自适应训练方法相比，本方法优越性明显，并能够改善词汇多样性。