One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted domains, or are low quality because they are constructed using semi-automatic procedures. In this work, we introduce the FLORES-101 evaluation benchmark, consisting of 3001 sentences extracted from English Wikipedia and covering a variety of different topics and domains. These sentences have been translated in 101 languages by professional translators through a carefully controlled process. The resulting dataset enables better assessment of model quality on the long tail of low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all translations are multilingually aligned. By publicly releasing such a high-quality and high-coverage dataset, we hope to foster progress in the machine translation community and beyond.

本文介绍了FLORES-101评估基准，该数据集由3001句从英文维基百科中提取出来的涵盖各种主题和领域的句子，在101种语言中由专业翻译人员进行翻译。该数据集可用于评估低资源语言和多对多多语言翻译系统的模型质量。

FLORES-101低资源和多语言机器翻译评估基准