Machine Translation Quality Estimation (MTQE) is the task of estimating the
quality of machine-translated text in real time without the need for reference
translations, which is of great importance for the development of MT. After two
decades of evolution, QE has yielded a wealth of results. This article provides
a comprehensive overview of QE datasets, annotation methods, shared tasks,
methodologies, challenges, and future research directions. It begins with an
introduction to the background and significance of QE, followed by an
explanation of the concepts and evaluation metrics for word-level QE,
sentence-level QE, document-level QE, and explainable QE. The paper categorizes
the methods developed throughout the history of QE into those based on
handcrafted features, deep learning, and Large Language Models (LLMs), with a
further division of deep learning-based methods into classic deep learning and
those incorporating pre-trained language models (LMs). Additionally, the
article details the advantages and limitations of each method and offers a
straightforward comparison of different approaches. Finally, the paper
discusses the current challenges in QE research and provides an outlook on
future research directions.

机器翻译质量评估（MTQE）是实时估计机器翻译文本质量的任务，不需要参考翻译，对机器翻译的发展非常重要。本文综述了质量评估数据集、标注方法、共享任务、方法学、挑战和未来研究方向。

从手工特征到 LLMs：机器翻译质量估计的简要调查

From Handcrafted Features to LLMs: A Brief Survey for Machine  Translation Quality Estimation

Machine Translation Quality Estimation (QE) is a task of predicting the
quality of machine translations without relying on any reference. Recently, the
predictor-estimator framework trains the predictor as a feature extractor,
which leverages the extra parallel corpora without QE labels, achieving
promising QE performance. However, we argue that there are gaps between the
predictor and the estimator in both data quality and training objectives, which
preclude QE models from benefiting from a large number of parallel corpora more
directly. We propose a novel framework called DirectQE that provides a direct
pretraining for QE tasks. In DirectQE, a generator is trained to produce pseudo
data that is closer to the real QE data, and a detector is pretrained on these
data with novel objectives that are akin to the QE task. Experiments on widely
used benchmarks show that DirectQE outperforms existing methods, without using
any pretraining models such as BERT. We also give extensive analyses showing
how fixing the two gaps contributes to our improvements.

本文提出 DirectQE 框架，通过直接预训练来提高机器翻译质量估计（QE）任务的性能，其中生成器训练产生更接近真实 QE 数据的伪数据，检测器使用类似 QE 任务的新目标在这些数据上进行预先训练。在广泛使用的基准测试中，DirectQE 胜过现有方法，而且没有使用任何预训练模型，如 BERT。

DirectQE: 机器翻译质量估计的直接预训练

DirectQE: Direct Pretraining for Machine Translation Quality Estimation

We present MLQE-PE, a new dataset for Machine Translation (MT) Quality
Estimation (QE) and Automatic Post-Editing (APE). The dataset contains eleven
language pairs, with human labels for up to 10,000 translations per language
pair in the following formats: sentence-level direct assessments and
post-editing effort, and word-level good/bad labels. It also contains the
post-edited sentences, as well as titles of the articles where the sentences
were extracted from, and the neural MT models used to translate the text.

MLQE-PE 为机器翻译质量评估和自动后编辑提供了新数据集，包含 11 种语言对，针对每种语言对的 10000 个翻译提供了人类标签，包括句子级别的直接评估和后编辑努力以及单词级别的好 / 坏标签，同时还提供了后编辑的句子、标题以及用于翻译文本的神经机器翻译模型。