This work introduces a simple regressive ensemble for evaluating machine
translation quality based on a set of novel and established metrics. We
evaluate the ensemble using a correlation to expert-based MQM scores of the WMT
2021 Metrics workshop. In both monolingual and zero-shot cross-lingual
settings, we show a significant performance improvement over single metrics. In
the cross-lingual settings, we also demonstrate that an ensemble approach is
well-applicable to unseen languages. Furthermore, we identify a strong
reference-free baseline that consistently outperforms the commonly-used BLEU
and METEOR measures and significantly improves our ensemble's performance.

本研究介绍了一种简单的回归集成方法，用于评估机器翻译质量。我们使用新颖和已建立的多种指标对集成模型进行了评估，并将其与 WMT 2021 Metrics 工作坊的专家评分进行了相关性比较。在单语和零 - shot 跨语言设置中，我们表明与单一指标相比，集成模型在性能上得到了显著的提高。在跨语言设置中，我们还证明了集成方法适用于未知语言。此外，我们还确定了一个强有力的基准线，它一直表现优于常用的 BLEU 和 METEOR 度量标准，并显着提高了我们集成模型的性能。

回归集成用于机器翻译质量评估

Regressive Ensemble for Machine Translation Quality Evaluation

Several neural-based metrics have been recently proposed to evaluate machine
translation quality. However, all of them resort to point estimates, which
provide limited information at segment level. This is made worse as they are
trained on noisy, biased and scarce human judgements, often resulting in
unreliable quality predictions. In this paper, we introduce uncertainty-aware
MT evaluation and analyze the trustworthiness of the predicted quality. We
combine the COMET framework with two uncertainty estimation methods, Monte
Carlo dropout and deep ensembles, to obtain quality scores along with
confidence intervals. We compare the performance of our uncertainty-aware MT
evaluation methods across multiple language pairs from the QT21 dataset and the
WMT20 metrics task, augmented with MQM annotations. We experiment with varying
numbers of references and further discuss the usefulness of uncertainty-aware
quality estimation (without references) to flag possibly critical translation
mistakes.

本研究介绍了一种基于神经网络度量的机器翻译质量不确定性评估方法，并结合蒙特卡罗 dropout 和深度集成等两种不确定度估计方法，得出质量分数以及置信区间。通过对来自 QT21 数据集和 WMT20 度量任务的多语种数据进行实验，验证了该方法的性能，进一步探讨了不依赖参考文献的不确定性评估在发现可能的翻译错误中的应用。

不确定性感知的机器翻译评估

Uncertainty-Aware Machine Translation Evaluation

Machine Translation Quality Estimation is a notoriously difficult task, which
lessens its usefulness in real-world translation environments. Such scenarios
can be improved if quality predictions are accompanied by a measure of
uncertainty. However, models in this task are traditionally evaluated only in
terms of point estimate metrics, which do not take prediction uncertainty into
account. We investigate probabilistic methods for Quality Estimation that can
provide well-calibrated uncertainty estimates and evaluate them in terms of
their full posterior predictive distributions. We also show how this posterior
information can be useful in an asymmetric risk scenario, which aims to capture
typical situations in translation workflows.

探索概率方法来提高机器翻译质量评估，提供良好的置信度估计，并通过全后验预测分布进行评估。还展示了在不对称风险的情况下如何利用后验信息，从而捕捉翻译工作流程中的典型情况。