This paper describes CAiRE's submission to the unsupervised machine translation track of the WMT'19 news shared task from German to Czech. We leverage a phrase-based statistical machine translation (PBSMT) model and a pre-trained language model to combine word-level neural machine translation (NMT) and subword-level NMT models without using any parallel data. We propose to solve the morphological richness problem of languages by training byte-pair encoding (BPE) embeddings for German and Czech separately, and they are aligned using MUSE (Conneau et al., 2018). To ensure the fluency and consistency of translations, a rescoring mechanism is proposed that reuses the pre-trained language model to select the translation candidates generated through beam search. Moreover, a series of pre-processing and post-processing approaches are applied to improve the quality of final translations.

本文讲述了CAiRE团队利用基于短语的统计机器翻译模型、预训练语言模型、字节对编码嵌入以及MUSE对德语和捷克语进行的非监督机器翻译，同时提出了重新评分机制以及对翻译的预处理和后处理方法以提高最终的翻译质量。

利用语言模型重打分，在无监督机器翻译中加入单词和子词单元