This paper describes Unbabel's submission to the WMT2019 APE Shared Task for
the English-German language pair. Following the recent rise of large, powerful,
pre-trained models, we adapt the BERT pretrained model to perform Automatic
Post-Editing in an encoder-decoder framework. Analogously to dual-encoder
architectures we develop a BERT-based encoder-decoder (BED) model in which a
single pretrained BERT encoder receives both the source src and machine
translation tgt strings. Furthermore, we explore a conservativeness factor to
constrain the APE system to perform fewer edits. As the official results show,
when trained on a weighted combination of in-domain and artificial training
data, our BED system with the conservativeness penalty improves significantly
the translations of a strong Neural Machine Translation system by $-0.78$ and
$+1.23$ in terms of TER and BLEU, respectively. Finally, our submission
achieves a new state-of-the-art, ex-aequo, in English-German APE of NMT.

针对英德翻译领域，Unbabel 利用 BERT 预训练模型，开发了基于 BERT 的编码器解码器模型 (BED)，通过对保守性因子的研究，将其与神经机器翻译系统相结合，使得翻译质量大幅提高，成为英德自动后编辑领域新的最高水平。