Despite the rising prevalence of neural sequence models, recent empirical
evidences suggest their deficiency in compositional generalization. One of the
current de-facto solutions to this problem is compositional data augmentation,
aiming to incur additional compositional inductive bias. Nonetheless, the
improvement offered by existing handcrafted augmentation strategies is limited
when successful systematic generalization of neural sequence models requires
multi-grained compositional bias (i.e., not limited to either lexical or
structural biases only) or differentiation of training sequences in an
imbalanced difficulty distribution. To address the two challenges, we first
propose a novel compositional augmentation strategy dubbed \textbf{Span}
\textbf{Sub}stitution (SpanSub) that enables multi-grained composition of
substantial substructures in the whole training set. Over and above that, we
introduce the \textbf{L}earning \textbf{to} \textbf{S}ubstitute \textbf{S}pan
(L2S2) framework which empowers the learning of span substitution probabilities
in SpanSub in an end-to-end manner by maximizing the loss of neural sequence
models, so as to outweigh those challenging compositions with elusive concepts
and novel surroundings. Our empirical results on three standard compositional
generalization benchmarks, including SCAN, COGS and GeoQuery (with an
improvement of at most 66.5\%, 10.3\%, 1.2\%, respectively), demonstrate the
superiority of SpanSub, %the learning framework L2S2 and their combination.

提出新的跨度替换策略（SpanSub）和基于嵌入式的学习框架（L2S2），分别解决了神经序列模型中的组合泛化问题和不均等难度分布下的挑战性组合问题，成功提高复杂数据集上的性能。