Zero-shot neural machine translation is an attractive goal because of the
high cost of obtaining data and building translation systems for new
translation directions. However, previous papers have reported mixed success in
zero-shot translation. It is hard to predict in which settings it will be
effective, and what limits performance compared to a fully supervised system.
In this paper, we investigate zero-shot performance of a multilingual
EN$\leftrightarrow${FR,CS,DE,FI} system trained on WMT data. We find that
zero-shot performance is highly unstable and can vary by more than 6 BLEU
between training runs, making it difficult to reliably track improvements. We
observe a bias towards copying the source in zero-shot translation, and
investigate how the choice of subword segmentation affects this bias. We find
that language-specific subword segmentation results in less subword copying at
training time, and leads to better zero-shot performance compared to jointly
trained segmentation. A recent trend in multilingual models is to not train on
parallel data between all language pairs, but have a single bridge language,
e.g. English. We find that this negatively affects zero-shot translation and
leads to a failure mode where the model ignores the language tag and instead
produces English output in zero-shot directions. We show that this bias towards
English can be effectively reduced with even a small amount of parallel data in
some of the non-English pairs.

研究了多语言零 - shot 机器翻译中的稳定性问题，发现语言特定的次词分割会提高翻译性能，单独的中间语言反而降低零 - shot 翻译表现，但是 在 非英语语言配对中使用少量的平行数据可以有效减少对英语的偏见。