The quality of machine translation has increased remarkably over the past
years, to the degree that it was found to be indistinguishable from
professional human translation in a number of empirical investigations. We
reassess Hassan et al.'s 2018 investigation into Chinese to English news
translation, showing that the finding of human-machine parity was owed to
weaknesses in the evaluation design - which is currently considered best
practice in the field. We show that the professional human translations
contained significantly fewer errors, and that perceived quality in human
evaluation depends on the choice of raters, the availability of linguistic
context, and the creation of reference translations. Our results call for
revisiting current best practices to assess strong machine translation systems
in general and human-machine parity in particular, for which we offer a set of
recommendations based on our empirical findings.

研究发现 2018 年 Hassan 等人的评估设计有问题，导致其认为人工翻译和机器翻译有同等水平的结论不准确，因此需要重新审视现有的评估方法，并且提出了一系列改进建议。