Compositional generalization is a fundamental trait in humans, allowing us to
effortlessly combine known phrases to form novel sentences. Recent works have
claimed that standard seq-to-seq models severely lack the ability to
compositionally generalize. In this paper, we focus on one-shot primitive
generalization as introduced by the popular SCAN benchmark. We demonstrate that
modifying the training distribution in simple and intuitive ways enables
standard seq-to-seq models to achieve near-perfect generalization performance,
thereby showing that their compositional generalization abilities were
previously underestimated. We perform detailed empirical analysis of this
phenomenon. Our results indicate that the generalization performance of models
is highly sensitive to the characteristics of the training data which should be
carefully considered while designing such benchmarks in future.

本文针对基础的 seq-to-seq 模型缺乏组合概括能力的问题，重点讨论了使用一次性原语概括来增强这种能力的方法，发现通过修改训练数据的方式，能够使标准的 seq-to-seq 模型实现接近完美的概括能力表现，并且对该现象进行了详细的实证分析，指出了模型的概括性能对于训练数据的特征非常敏感，因而应该在设计这样的基准测试时仔细考虑训练数据的选择。