Translation Quality Estimation (QE) is the task of predicting the quality of machine translation (MT) output without any reference. This task has gained increasing attention as an important component in practical applications of MT. In this paper, we first propose XLMRScore, a simple unsupervised QE method based on the BERTScore computed using the XLM-RoBERTa (XLMR) model while discussing the issues that occur using this method. Next, we suggest two approaches to mitigate the issues: replacing untranslated words with the unknown token and the cross-lingual alignment of pre-trained model to represent aligned words closer to each other. We evaluate the proposed method on four low-resource language pairs of WMT21 QE shared task, as well as a new English-Farsi test dataset introduced in this paper. Experiments show that our method could get comparable results with the supervised baseline for two zero-shot scenarios, i.e., with less than 0.01 difference in Pearson correlation, while outperforming the unsupervised rivals in all the low-resource language pairs for above 8% in average.

本文提出了一种简单的无监督翻译质量评估方法XLMRScore，该方法基于使用XLM-RoBERTa模型计算的BertScore，并讨论了使用此方法时出现的问题。接着，我们提出两种方法来缓解问题，并将所提出的方法用于四个WMT21 QE shared task中的低资源语言对以及本文介绍的一个新的英语-波斯语测试数据集。实验表明，在两个零-shot场景下，我们的方法可以获得与有监督基线相当的结果，即Pearson相关性差异小于0.01，在所有低资源语言对中的表现均优于无监督对手，平均超过8％。

针对低资源语言的不匹配感知无监督翻译质量评估