We present COMET, a neural framework for training multilingual machine
translation evaluation models which obtains new state-of-the-art levels of
correlation with human judgements. Our framework leverages recent breakthroughs
in cross-lingual pretrained language modeling resulting in highly multilingual
and adaptable MT evaluation models that exploit information from both the
source input and a target-language reference translation in order to more
accurately predict MT quality. To showcase our framework, we train three models
with different types of human judgements: Direct Assessments, Human-mediated
Translation Edit Rate and Multidimensional Quality Metrics. Our models achieve
new state-of-the-art performance on the WMT 2019 Metrics shared task and
demonstrate robustness to high-performing systems.

介绍了 COMET，这是一个使用神经网络的多语言机器翻译评估模型，它利用了跨语言预训练语言建模的最新进展，通过同时使用源输入和目标语言的参考翻译来更准确地预测机器翻译的质量，并在 WMT 2019 指标共享任务中获得了新的最优表现，并展示了对高性能系统的鲁棒性。