Low-frequency word prediction remains a challenge in modern neural machine
translation (NMT) systems. Recent adaptive training methods promote the output
of infrequent words by emphasizing their weights in the overall training
objectives. Despite the improved recall of low-frequency words, their
prediction precision is unexpectedly hindered by the adaptive objectives.
Inspired by the observation that low-frequency words form a more compact
embedding space, we tackle this challenge from a representation learning
perspective. Specifically, we propose a frequency-aware token-level contrastive
learning method, in which the hidden state of each decoding step is pushed away
from the counterparts of other target words, in a soft contrastive way based on
the corresponding word frequencies. We conduct experiments on widely used NIST
Chinese-English and WMT14 English-German translation tasks. Empirical results
show that our proposed methods can not only significantly improve the
translation quality but also enhance lexical diversity and optimize word
representation space. Further investigation reveals that, comparing with
related adaptive training strategies, the superiority of our method on
low-frequency word prediction lies in the robustness of token-level recall
across different frequencies without sacrificing precision.

本文提出一种基于词频感知的令牌级对比学习方法，旨在从表示学习角度解决现代神经机器翻译系统中低频词预测的挑战。经实验证明，所提出的方法不仅可以显著提高翻译质量，还可以增加词汇多样性并优化词表示空间。与相关的自适应训练策略相比，该方法在不牺牲精度的前提下提高了低频词汇量的召回率稳健性。