Using a shared vocabulary is common practice in Multilingual Neural Machine
Translation (MNMT). In addition to its simple design, shared tokens play an
important role in positive knowledge transfer, which manifests naturally when
the shared tokens refer to similar meanings across languages. However, natural
flaws exist in such a design as well: 1) when languages use different writing
systems, transfer is inhibited, and 2) even if languages use similar writing
systems, shared tokens may have completely different meanings in different
languages, increasing ambiguity. In this paper, we propose a re-parameterized
method for building embeddings to alleviate the first problem. More
specifically, we define word-level information transfer pathways via word
equivalence classes and rely on graph networks to fuse word embeddings across
languages. Our experiments demonstrate the advantages of our approach: 1) the
semantics of embeddings are better aligned across languages, 2) our method
achieves significant BLEU improvements on high- and low-resource MNMT, and 3)
only less than 1.0\% additional trainable parameters are required with a
limited increase in computational costs.

本论文提出了一种新的共享词汇的设计方法，通过定义词级别信息传递路径和使用图网络来融合跨语言的词嵌入，避免了多语言之间文字编码的问题，实验还表明该方法在高低资源多语言机器翻译中可实现显著的 BLEU 分数改进，且只需额外训练少于 1.0％的可训练参数，计算成本增加有限。

跨语言机器翻译中的词汇表不足：增加语言间表示单词的相似度

Beyond Shared Vocabulary: Increasing Representational Word Similarities  across Languages for Multilingual Machine Translation

Multilingual neural machine translation models are trained to maximize the
likelihood of a mix of examples drawn from multiple language pairs. The
dominant inductive bias applied to these models is a shared vocabulary and a
shared set of parameters across languages; the inputs and labels corresponding
to examples drawn from different language pairs might still reside in distinct
sub-spaces. In this paper, we introduce multilingual crossover encoder-decoder
(mXEncDec) to fuse language pairs at an instance level. Our approach
interpolates instances from different language pairs into joint `crossover
examples' in order to encourage sharing input and output spaces across
languages. To ensure better fusion of examples in multilingual settings, we
propose several techniques to improve example interpolation across dissimilar
languages under heavy data imbalance. Experiments on a large-scale WMT
multilingual dataset demonstrate that our approach significantly improves
quality on English-to-Many, Many-to-English and zero-shot translation tasks
(from +0.5 BLEU up to +5.5 BLEU points). Results on code-switching sets
demonstrate the capability of our approach to improve model generalization to
out-of-distribution multilingual examples. We also conduct qualitative and
quantitative representation comparisons to analyze the advantages of our
approach at the representation level.

本文介绍了一种多语言交叉编码解码器 (mXEncDec) 方法，以在实例级别融合多种语言对，通过改进实例插值技术跨多种语言改善 WMT 数据集上的机器翻译效果，并证明其能够改善模型的推广性。

多语言混合：示例插值改进多语言神经机器翻译

Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation

We propose a simple solution to use a single Neural Machine Translation (NMT)
model to translate between multiple languages. Our solution requires no change
in the model architecture from our base system but instead introduces an
artificial token at the beginning of the input sentence to specify the required
target language. The rest of the model, which includes encoder, decoder and
attention, remains unchanged and is shared across all languages. Using a shared
wordpiece vocabulary, our approach enables Multilingual NMT using a single
model without any increase in parameters, which is significantly simpler than
previous proposals for Multilingual NMT. Our method often improves the
translation quality of all involved language pairs, even while keeping the
total number of model parameters constant. On the WMT'14 benchmarks, a single
multilingual model achieves comparable performance for
English$\rightarrow$French and surpasses state-of-the-art results for
English$\rightarrow$German. Similarly, a single multilingual model surpasses
state-of-the-art results for French$\rightarrow$English and
German$\rightarrow$English on WMT'14 and WMT'15 benchmarks respectively. On
production corpora, multilingual models of up to twelve language pairs allow
for better translation of many individual pairs. In addition to improving the
translation quality of language pairs that the model was trained with, our
models can also learn to perform implicit bridging between language pairs never
seen explicitly during training, showing that transfer learning and zero-shot
translation is possible for neural translation. Finally, we show analyses that
hints at a universal interlingua representation in our models and show some
interesting examples when mixing languages.

该研究提出一种简单的解决方案，使用单个神经机器翻译模型在多种语言之间进行翻译，并且通过在输入句子的开头引入人工标记来指定所需的目标语言，这种方法不需要更改模型框架，该模型的剩余组件包括编码器、解码器和注意力是不变的，并共享所有语言。我们的方法使用共享的词块词汇表，不需要增加任何参数，在保持模型参数总数恒定的情况下，还经常提高所有涉及的语言对的翻译质量，甚至可以在训练期间从未看到的语言对之间进行隐式桥接，因此，我们的翻译模型不限于训练时的语言对，具有一定的通用性和迁移能力。