Lexical ambiguity is a significant and pervasive challenge in Neural Machine
Translation (NMT), with many state-of-the-art (SOTA) NMT systems struggling to
handle polysemous words (Campolungo et al., 2022). The same holds for the NMT
pretraining paradigm of denoising synthetic "code-switched" text (Pan et al.,
2021; Iyer et al., 2023), where word senses are ignored in the noising stage --
leading to harmful sense biases in the pretraining data that are subsequently
inherited by the resulting models. In this work, we introduce Word Sense
Pretraining for Neural Machine Translation (WSP-NMT) - an end-to-end approach
for pretraining multilingual NMT models leveraging word sense-specific
information from Knowledge Bases. Our experiments show significant improvements
in overall translation quality. Then, we show the robustness of our approach to
scale to various challenging data and resource-scarce scenarios and, finally,
report fine-grained accuracy improvements on the DiBiMT disambiguation
benchmark. Our studies yield interesting and novel insights into the merits and
challenges of integrating word sense information and structured knowledge in
multilingual pretraining for NMT.

我们介绍了一种使用知识库中词义特定信息进行预训练的多语言神经机器翻译模型的端到端方法，通过实验证明了我们方法在整体翻译质量上有显著改进，并且在 DiBiMT 消歧测试中显示出了精确性改进的鲁棒性。