The rapid development of large pretrained language models has revolutionized
not only the field of Natural Language Generation (NLG) but also its
evaluation. Inspired by the recent work of BARTScore: a metric leveraging the
BART language model to evaluate the quality of generated text from various
aspects, we introduce DATScore. DATScore uses data augmentation techniques to
improve the evaluation of machine translation. Our main finding is that
introducing data augmented translations of the source and reference texts is
greatly helpful in evaluating the quality of the generated translation. We also
propose two novel score averaging and term weighting strategies to improve the
original score computing process of BARTScore. Experimental results on WMT show
that DATScore correlates better with human meta-evaluations than the other
recent state-of-the-art metrics, especially for low-resource languages.
Ablation studies demonstrate the value added by our new scoring strategies.
Moreover, we report in our extended experiments the performance of DATScore on
3 NLG tasks other than translation.

本论文介绍了一种使用数据增强技术来改进机器翻译评估的 DATScore 指标，并提出两种新的评分和术语加权策略以改善 BARTScore 计算过程的论点。实证结果表明，DATScore 与人类元评价的相关性比其他最新的最先进度量标准更好，特别是对于低资源语言。

DATScore：使用数据增强的翻译评估方法

DATScore: Evaluating Translation with Data Augmented Translations

A wide variety of NLP applications, such as machine translation,
summarization, and dialog, involve text generation. One major challenge for
these applications is how to evaluate whether such generated texts are actually
fluent, accurate, or effective. In this work, we conceptualize the evaluation
of generated text as a text generation problem, modeled using pre-trained
sequence-to-sequence models. The general idea is that models trained to convert
the generated text to/from a reference output or the source text will achieve
higher scores when the generated text is better. We operationalize this idea
using BART, an encoder-decoder based pre-trained model, and propose a metric
BARTScore with a number of variants that can be flexibly applied in an
unsupervised fashion to evaluation of text from different perspectives (e.g.
informativeness, fluency, or factuality). BARTScore is conceptually simple and
empirically effective. It can outperform existing top-scoring metrics in 16 of
22 test settings, covering evaluation of 16 datasets (e.g., machine
translation, text summarization) and 7 different perspectives (e.g.,
informativeness, factuality). Code to calculate BARTScore is available at
this https URL, and we have released an interactive
leaderboard for meta-evaluation at
this http URL on the ExplainaBoard
platform, which allows us to interactively understand the strengths,
weaknesses, and complementarity of each metric.

本研究利用预训练的序列到序列模型建立生成文本评估系统，并提出了一种与可靠的度量标准 BARTScore，可灵活应用于不同角度的文本评估，包括信息性，流畅性和事实性等领域，通过在 16 个数据集和 7 个不同角度的测试中超越其他评测度量标准。