We study the utility of incorporating entity type abstractions into
pre-trained Transformers and test these methods on four NLP tasks requiring
different forms of logical reasoning: (1) compositional language understanding
with text-based relational reasoning (CLUTRR), (2) abductive reasoning
(ProofWriter), (3) multi-hop question answering (HotpotQA), and (4)
conversational question answering (CoQA). We propose and empirically explore
three ways to add such abstraction: (i) as additional input embeddings, (ii) as
a separate sequence to encode, and (iii) as an auxiliary prediction task for
the model. Overall, our analysis demonstrates that models with abstract entity
knowledge performs better than without it. The best abstraction aware models
achieved an overall accuracy of 88.8% and 91.8% compared to the baseline model
achieving 62.9% and 89.8% on CLUTRR and ProofWriter respectively. However, for
HotpotQA and CoQA, we find that F1 scores improve by only 0.5% on average. Our
results suggest that the benefit of explicit abstraction is significant in
formally defined logical reasoning settings requiring many reasoning hops, but
point to the notion that it is less beneficial for NLP tasks having less formal
logical structure.

研究使用实体类型抽象在预训练的转换器中的效用，并在四个需要不同形式逻辑推理的自然语言处理任务上测试这些方法：(1) 基于文本关系推理的组合语言理解 (CLUTRR)，(2) 异化推理 (ProofWriter)，(3) 多跳问题回答 (HotpotQA) 和 (4) 对话问题回答 (CoQA)。我们提出并实证探讨了三种添加这种抽象的方法：(i) 作为额外的输入嵌入，(ii) 作为单独的序列来编码，(iii) 作为模型的辅助预测任务。总的来说，我们的分析表明，具有实体抽象知识的模型比没有它的模型表现得更好。最佳的抽象感知模型在 CLUTRR 和 ProofWriter 上实现了 88.8% 和 91.8% 的总体准确性，而基线模型分别实现了 62.9% 和 89.8%。然而，对于 HotpotQA 和 CoQA，我们发现 F1 得分平均只提高了 0.5%。我们的结果表明，显式抽象的好处在形式上定义了需要多种推理跳数的逻辑推理设置中是显著的，但指向了这种抽象在较少准确逻辑结构的 NLP 任务中益处较小的概念。