Sequence-to-sequence (seq2seq) models have been successful across many NLP
tasks, including ones that require predicting linguistic structure. However,
recent work on compositional generalization has shown that seq2seq models
achieve very low accuracy in generalizing to linguistic structures that were
not seen in training. We present new evidence that this is a general limitation
of seq2seq models that is present not just in semantic parsing, but also in
syntactic parsing and in text-to-text tasks, and that this limitation can often
be overcome by neurosymbolic models that have linguistic knowledge built in. We
further report on some experiments that give initial answers on the reasons for
these limitations.

通过实验证明，Seq2Seq 模型在英语句法分析、语义分析和文本之间的转换任务中，普遍存在推广的普适性较低的问题，但是在建立了语言知识的神经符号模型中，这些限制往往可以克服。

序列到序列模型对结构泛化的困难

Structural generalization is hard for sequence-to-sequence models

QuestEval is a reference-less metric used in text-to-text tasks, that
compares the generated summaries directly to the source text, by automatically
asking and answering questions. Its adaptation to Data-to-Text tasks is not
straightforward, as it requires multimodal Question Generation and Answering
systems on the considered tasks, which are seldom available. To this purpose,
we propose a method to build synthetic multimodal corpora enabling to train
multimodal components for a data-QuestEval metric. The resulting metric is
reference-less and multimodal; it obtains state-of-the-art correlations with
human judgment on the WebNLG and WikiBio benchmarks. We make data-QuestEval's
code and models available for reproducibility purpose, as part of the QuestEval
project.

QuestEval 是一种用于文本 - to - 文本任务的无参考度量标准，它通过自动提问和回答直接将生成的摘要与源文本进行比较。我们提出了一种方法来构建合成的多模态语料库，以训练多模态组件用于数据 - QuestEval 指标，并在 WebNLG 和 WikiBio 基准测试上获得了与人类判断的最新相关性。