Understanding representation transfer in multilingual neural machine
translation can reveal the representational issue causing the zero-shot
translation deficiency. In this work, we introduce the identity pair, a
sentence translated into itself, to address the lack of the base measure in
multilingual investigations, as the identity pair represents the optimal state
of representation among any language transfers. In our analysis, we demonstrate
that the encoder transfers the source language to the representational subspace
of the target language instead of the language-agnostic state. Thus, the
zero-shot translation deficiency arises because representations are entangled
with other languages and are not transferred effectively to the target
language. Based on our findings, we propose two methods: 1) low-rank
language-specific embedding at the encoder, and 2) language-specific
contrastive learning of the representation at the decoder. The experimental
results on Europarl-15, TED-19, and OPUS-100 datasets show that our methods
substantially enhance the performance of zero-shot translations by improving
language transfer capacity, thereby providing practical evidence to support our
conclusions.

这篇论文通过研究多语言神经机器翻译中的表示转移，揭示了导致零射翻译不足的表示问题。研究中引入了身份对（一个句子翻译为自身），以解决多语言研究中基准测量缺失的问题，因为身份对代表了所有语言转移中的最佳表示状态。通过分析，我们证明编码器将源语言转移到目标语言的表示子空间，而不是语言无关的状态。因此，零射翻译不足是因为表示与其他语言相混合并且未能有效地转移到目标语言。基于我们的发现，我们提出了两种方法：1）在编码器中使用低秩语言特定嵌入，2）在解码器中进行语言特定的对比学习表示。在 Europarl-15、TED-19 和 OPUS-100 数据集上的实验结果表明，我们的方法显著提高了零射翻译的性能，改善了语言转移能力，从而为我们的结论提供了实际证据。