Driven by the goal of eradicating language barriers on a global scale,
machine translation has solidified itself as a key focus of artificial
intelligence research today. However, such efforts have coalesced around a
small subset of languages, leaving behind the vast majority of mostly
low-resource languages. What does it take to break the 200 language barrier
while ensuring safe, high quality results, all while keeping ethical
considerations in mind? In No Language Left Behind, we took on this challenge
by first contextualizing the need for low-resource language translation support
through exploratory interviews with native speakers. Then, we created datasets
and models aimed at narrowing the performance gap between low and high-resource
languages. More specifically, we developed a conditional compute model based on
Sparsely Gated Mixture of Experts that is trained on data obtained with novel
and effective data mining techniques tailored for low-resource languages. We
propose multiple architectural and training improvements to counteract
overfitting while training on thousands of tasks. Critically, we evaluated the
performance of over 40,000 different translation directions using a
human-translated benchmark, Flores-200, and combined human evaluation with a
novel toxicity benchmark covering all languages in Flores-200 to assess
translation safety. Our model achieves an improvement of 44% BLEU relative to
the previous state-of-the-art, laying important groundwork towards realizing a
universal translation system. Finally, we open source all contributions
described in this work, accessible at
this https URL.

本文介绍了一项针对低资源语言的研究，使用 Sparsely Gated Mixture of Experts 模型结合新的数据挖掘技术进行训练，从而实现了机器翻译中对于低资源语言的支持并提高了 BLEU 值。

无人被落下的语言：人本机器翻译的扩展

No Language Left Behind: Scaling Human-Centered Machine Translation

We introduce our efforts towards building a universal neural machine
translation (NMT) system capable of translating between any language pair. We
set a milestone towards this goal by building a single massively multilingual
NMT model handling 103 languages trained on over 25 billion examples. Our
system demonstrates effective transfer learning ability, significantly
improving translation quality of low-resource languages, while keeping
high-resource language translation quality on-par with competitive bilingual
baselines. We provide in-depth analysis of various aspects of model building
that are crucial to achieving quality and practicality in universal NMT. While
we prototype a high-quality universal translation system, our extensive
empirical analysis exposes issues that need to be further addressed, and we
suggest directions for future research.

我们致力于构建一款通用的神经机器翻译系统，通过构建一个单一的大规模多语言 NMT 模型，实现了 103 种语言之间的翻译，带有有效的迁移学习能力，显着提高了低资源语言的翻译质量，同时保持高资源语言翻译质量与竞争双语基线相当，为实现通用 NMT 模型的质量和实用性提供了多个方面的模型构建分析，并指出未来研究的方向和需进一步解决的问题。