We introduce DivEMT, the first publicly available post-editing study of
Neural Machine Translation (NMT) over a typologically diverse set of target
languages. Using a strictly controlled setup, 18 professional translators were
instructed to translate or post-edit the same set of English documents into
Arabic, Dutch, Italian, Turkish, Ukrainian, and Vietnamese. During the process,
their edits, keystrokes, editing times and pauses were recorded, enabling an
in-depth, cross-lingual evaluation of NMT quality and post-editing
effectiveness. Using this new dataset, we assess the impact of two
state-of-the-art NMT systems, Google Translate and the multilingual mBART-50
model, on translation productivity. We find that post-editing is consistently
faster than translation from scratch. However, the magnitude of productivity
gains varies widely across systems and languages, highlighting major
disparities in post-editing effectiveness for languages at different degrees of
typological relatedness to English, even when controlling for system
architecture and training data size. We publicly release the complete dataset
including all collected behavioral data, to foster new research on the
translation capabilities of NMT systems for typologically diverse languages.

本研究针对多种语言数十款神经机器翻译模型，通过一项系统性的翻译后修订实验，对每种翻译成果进行了专业化评估，并公开了完整数据集，旨在进一步研究神经机器翻译系统的翻译能力。