Fast and reliable evaluation metrics are key to R&D progress. While traditional natural language generation metrics are fast, they are not very reliable. Conversely, new metrics based on large pretrained language models are much more reliable, but require significant computational resources. In this paper, we propose FrugalScore, an approach to learn a fixed, low cost version of any expensive NLG metric, while retaining most of its original performance. Experiments with BERTScore and MoverScore on summarization and translation show that FrugalScore is on par with the original metrics (and sometimes better), while having several orders of magnitude less parameters and running several times faster. On average over all learned metrics, tasks, and variants, FrugalScore retains 96.8% of the performance, runs 24 times faster, and has 35 times less parameters than the original metrics. We make our trained metrics publicly available, to benefit the entire NLP community and in particular researchers and practitioners with limited resources.

本文提出了一种名为FrugalScore的方法，用于学习定制的低成本版本的自然语言生成度量标准，其性能与原始的度量标准相当，甚至更好，同时具有更少的参数和更快的运行速度，为有限资源的研究者和从业者提供帮助。

FrugalScore：学习更便宜、更轻、更快的自动文本生成评估指标