Human intelligence exhibits compositional generalization (i.e., the capacity
to understand and produce unseen combinations of seen components), but current
neural seq2seq models lack such ability. In this paper, we revisit iterative
back-translation, a simple yet effective semi-supervised method, to investigate
whether and how it can improve compositional generalization. In this work: (1)
We first empirically show that iterative back-translation substantially
improves the performance on compositional generalization benchmarks (CFQ and
SCAN). (2) To understand why iterative back-translation is useful, we carefully
examine the performance gains and find that iterative back-translation can
increasingly correct errors in pseudo-parallel data. (3) To further encourage
this mechanism, we propose curriculum iterative back-translation, which better
improves the quality of pseudo-parallel data, thus further improving the
performance.

本研究通过迭代回译一种简单而有效的半监督方法，证实其对于复合泛化的基准数据（CFQ 和 SCAN）的性能显著提高，并为理解这一机制提出了课程迭代回译，进一步提高伪并行数据的质量，进一步改善性能。

从组合概括的角度重新审视迭代回译

Revisiting Iterative Back-Translation from the Perspective of  Compositional Generalization

While Iterative Back-Translation and Dual Learning effectively incorporate
monolingual training data in neural machine translation, they use different
objectives and heuristic gradient approximation strategies, and have not been
extensively compared. We introduce a novel dual reconstruction objective that
provides a unified view of Iterative Back-Translation and Dual Learning. It
motivates a theoretical analysis and controlled empirical study on
German-English and Turkish-English tasks, which both suggest that Iterative
Back-Translation is more effective than Dual Learning despite its relative
simplicity.

比较了交替回译和双向学习的效果，提出了新的双重重建目标，实验证明对于德英和土英任务，交替回译比双向学习更加有效。

双重重建：半监督神经机器翻译的统一目标

Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural  Machine Translation

Recent unsupervised machine translation (UMT) systems usually employ three
main principles: initialization, language modeling and iterative
back-translation, though they may apply them differently. Crucially, iterative
back-translation and denoising auto-encoding for language modeling provide data
diversity to train the UMT systems. However, the gains from these
diversification processes has seemed to plateau. We introduce a novel component
to the standard UMT framework called Cross-model Back-translated Distillation
(CBD), that is aimed to induce another level of data diversification that
existing principles lack. CBD is applicable to all previous UMT approaches. In
our experiments, CBD achieves the state of the art in the WMT'14
English-French, WMT'16 English-German and English-Romanian bilingual
unsupervised translation tasks, with 38.2, 30.1, and 36.3 BLEU respectively. It
also yields 1.5-3.3 BLEU improvements in IWSLT English-French and
English-German tasks. Through extensive experimental analyses, we show that CBD
is effective because it embraces data diversity while other similar variants do
not.

本文介绍一种新的 UMT 框架组成部分叫做跨模型反向传递蒸馏（CBD）来促进数据多样性，与其他变体相比，CBD 更加有效