Recent developments in machine translation and multilingual text generation
have led researchers to adopt trained metrics such as COMET or BLEURT, which
treat evaluation as a regression problem and use representations from
multilingual pre-trained models such as XLM-RoBERTa or mBERT. Yet studies on
related tasks suggest that these models are most efficient when they are large,
which is costly and impractical for evaluation. We investigate the trade-off
between multilinguality and model capacity with RemBERT, a state-of-the-art
multilingual language model, using data from the WMT Metrics Shared Task. We
present a series of experiments which show that model size is indeed a
bottleneck for cross-lingual transfer, then demonstrate how distillation can
help addressing this bottleneck, by leveraging synthetic data generation and
transferring knowledge from one teacher to multiple students trained on related
languages. Our method yields up to 10.5% improvement over vanilla fine-tuning
and reaches 92.6% of RemBERT's performance using only a third of its
parameters.

使用 RemBERT 模型的实验表明，模型大小限制了跨语言转换的效率，将知识从一名教师传递到以相关语言训练的多个学生可以通过蒸馏解决这个问题，该方法可以使性能提高 10.5％，并仅使用 RemBERT 的三分之一的参数即可达到 92.6％的性能。

为机器翻译学习紧凑的度量

Learning Compact Metrics for MT

We introduce MTG, a new benchmark suite for training and evaluating
multilingual text generation. It is the first-proposed multilingual multiway
text generation dataset with the largest human-annotated data (400k). It
includes four generation tasks (story generation, question generation, title
generation and text summarization) across five languages (English, German,
French, Spanish and Chinese). The multiway setup enables testing knowledge
transfer capabilities for a model across languages and tasks. Using MTG, we
train and analyze several popular multilingual generation models from different
aspects. Our benchmark suite fosters model performance enhancement with more
human-annotated parallel data. It provides comprehensive evaluations with
diverse generation scenarios. Code and data are available at
https://github.com/zide05/MTG.

介绍了一个新的基准测试套件，MTG，用于训练和评估多语言文本生成。该套件包括四个生成任务（故事生成、问题生成、标题生成和文本摘要）和五种语言，可测试模型在不同语言和任务之间的知识转移能力，有利于提高模型性能，并提供多样化的生成方案的全面评估。