Providing quality scores along with Machine Translation (MT) output,
so-called reference-free Quality Estimation (QE), is crucial to inform users
about the reliability of the translation. We propose a model-specific,
unsupervised QE approach, termed $k$NN-QE, that extracts information from the
MT model's training data using $k$-nearest neighbors. Measuring the performance
of model-specific QE is not straightforward, since they provide quality scores
on their own MT output, thus cannot be evaluated using benchmark QE test sets
containing human quality scores on premade MT output. Therefore, we propose an
automatic evaluation method that uses quality scores from reference-based
metrics as gold standard instead of human-generated ones. We are the first to
conduct detailed analyses and conclude that this automatic method is
sufficient, and the reference-based MetricX-23 is best for the task.

我们提出了一种模型特定的无监督质量评估方法，称为 $k$NN-QE，该方法从机器翻译模型的训练数据中提取信息，并使用 $k$ 个最近邻进行评估。我们还提出了一种使用基于参考的度量标准的质量评估自动方法，并通过详细分析得出结论，MetricX-23 是最适合该任务的方法。