Compositional generalization is a basic mechanism in human language learning,
which current neural networks struggle with. A recently proposed Disentangled
sequence-to-sequence model (Dangle) shows promising generalization capability
by learning specialized encodings for each decoding step. We introduce two key
modifications to this model which encourage more disentangled representations
and improve its compute and memory efficiency, allowing us to tackle
compositional generalization in a more realistic setting. Specifically, instead
of adaptively re-encoding source keys and values at each time step, we
disentangle their representations and only re-encode keys periodically, at some
interval. Our new architecture leads to better generalization performance
across existing tasks and datasets, and a new machine translation benchmark
which we create by detecting naturally occurring compositional patterns in
relation to a training set. We show this methodology better emulates real-world
requirements than artificial challenges.

通过对 Disentangled sequence-to-sequence 模型进行改进，我们引入了两个关键的修改，以更好地处理语言中的组合概括，从而在现有任务和数据集上实现更好的泛化性能，并创造了一个新的机器翻译基准，并展示了这种方法比人工设计呈现更好的真实通用性。