Back-translation is a critical component of Unsupervised Neural Machine
Translation (UNMT), which generates pseudo parallel data from target
monolingual data. A UNMT model is trained on the pseudo parallel data with
translated source, and translates natural source sentences in inference. The
source discrepancy between training and inference hinders the translation
performance of UNMT models. By carefully designing experiments, we identify two
representative characteristics of the data gap in source: (1) style gap (i.e.,
translated vs. natural text style) that leads to poor generalization
capability; (2) content gap that induces the model to produce hallucination
content biased towards the target language. To narrow the data gap, we propose
an online self-training approach, which simultaneously uses the pseudo parallel
data {natural source, translated target} to mimic the inference scenario.
Experimental results on several widely-used language pairs show that our
approach outperforms two strong baselines (XLM and MASS) by remedying the style
and content gaps.

通过在线自我训练方法缩小数据差距，使用伪并行数据来模拟推理情景，从而解决了 Unsupervised Neural Machine Translation​中的源数据差异所带来的挑战。

无监督神经机器翻译训练和推断之间数据差距的纾解

Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translation

We propose a novel data synthesis method to generate diverse error-corrected
sentence pairs for improving grammatical error correction, which is based on a
pair of machine translation models of different qualities (i.e., poor and
good). The poor translation model resembles the ESL (English as a second
language) learner and tends to generate translations of low quality in terms of
fluency and grammatical correctness, while the good translation model generally
generates fluent and grammatically correct translations. We build the poor and
good translation model with phrase-based statistical machine translation model
with decreased language model weight and neural machine translation model
respectively. By taking the pair of their translations of the same sentences in
a bridge language as error-corrected sentence pairs, we can construct unlimited
pseudo parallel data. Our approach is capable of generating diverse
fluency-improving patterns without being limited by the pre-defined rule set
and the seed error-corrected data. Experimental results demonstrate the
effectiveness of our approach and show that it can be combined with other
synthetic data sources to yield further improvements.

提出了一种新颖的数据合成方法，以生成多样的句子对，用于改善语法错误纠正，该方法基于两个不同质量（即较差和较好）的机器翻译模型对。实验结果表明，我们的方法是有效的，可以与其他合成数据源相结合，以产生进一步的改进。