Social Commonsense Reasoning requires understanding of text, knowledge about
social events and their pragmatic implications, as well as commonsense
reasoning skills. In this work we propose a novel multi-head knowledge
attention model that encodes semi-structured commonsense inference rules and
learns to incorporate them in a transformer-based reasoning cell. We assess the
model's performance on two tasks that require different reasoning skills:
Abductive Natural Language Inference and Counterfactual Invariance Prediction
as a new task. We show that our proposed model improves performance over strong
state-of-the-art models (i.e., RoBERTa) across both reasoning tasks. Notably we
are, to the best of our knowledge, the first to demonstrate that a model that
learns to perform counterfactual reasoning helps predicting the best
explanation in an abductive reasoning task. We validate the robustness of the
model's reasoning capabilities by perturbing the knowledge and provide
qualitative analysis on the model's knowledge incorporation capabilities.

本文提出了一种新颖的多头知识注意模型，该模型对半结构化的常识推理规则进行编码，并学习将它们纳入基于 Transformer 的推理单元中。在推理任务中，我们评估了模型在需要不同推理技能的两个任务上的表现，即推断性自然语言推理和反事实不变性预测，并展示了我们提出的模型相对于强大的最先进模型（例如 RoBERTa）在两个推理任务中的表现提高，通过扰动知识，并对模型的知识纳入能力进行定性分析，我们验证了模型推理能力的稳健性。值得注意的是，我们在最佳解释方面展示了学习执行反事实推理的模型对推断性推理任务的帮助，这在我们所知道的情况下是首次演示的。

多头知识关注的社交常识推理

Social Commonsense Reasoning with Multi-Head Knowledge Attention

The abductive natural language inference task ($\alpha$NLI) is proposed to
evaluate the abductive reasoning ability of a learning system. In the
$\alpha$NLI task, two observations are given and the most plausible hypothesis
is asked to pick out from the candidates. Existing methods simply formulate it
as a classification problem, thus a cross-entropy log-loss objective is used
during training. However, discriminating true from false does not measure the
plausibility of a hypothesis, for all the hypotheses have a chance to happen,
only the probabilities are different. To fill this gap, we switch to a ranking
perspective that sorts the hypotheses in order of their plausibilities. With
this new perspective, a novel $L2R^2$ approach is proposed under the
learning-to-rank framework. Firstly, training samples are reorganized into a
ranking form, where two observations and their hypotheses are treated as the
query and a set of candidate documents respectively. Then, an ESIM model or
pre-trained language model, e.g. BERT or RoBERTa, is obtained as the scoring
function. Finally, the loss functions for the ranking task can be either
pair-wise or list-wise for training. The experimental results on the ART
dataset reach the state-of-the-art in the public leaderboard.

本文提出了一种基于排序的 $L2R^2$ 方法，用于解决自然语言推理任务中的假设排序问题，并使用 ESIM 或预训练语言模型（如 BERT 或 RoBERTa）作为评分函数，实验证明，该方法取得了 ART 数据集中最先进的性能水平。