Providing quality scores along with Machine Translation (MT) output, so-called reference-free Quality Estimation (QE), is crucial to inform users about the reliability of the translation. We propose a model-specific, unsupervised QE approach, termed $k$NN-QE, that extracts information from the MT model's training data using $k$-nearest neighbors. Measuring the performance of model-specific QE is not straightforward, since they provide quality scores on their own MT output, thus cannot be evaluated using benchmark QE test sets containing human quality scores on premade MT output. Therefore, we propose an automatic evaluation method that uses quality scores from reference-based metrics as gold standard instead of human-generated ones. We are the first to conduct detailed analyses and conclude that this automatic method is sufficient, and the reference-based MetricX-23 is best for the task.

我们提出了一种模型特定的无监督质量评估方法，称为$k$NN-QE，该方法从机器翻译模型的训练数据中提取信息，并使用$k$个最近邻进行评估。我们还提出了一种使用基于参考的度量标准的质量评估自动方法，并通过详细分析得出结论，MetricX-23是最适合该任务的方法。

基于$k$最近邻和自动评估的质量估计与特定模型质量估计