While pre-trained language models (PLMs) are the go-to solution to tackle
many natural language processing problems, they are still very limited in their
ability to capture and to use common-sense knowledge. In fact, even if
information is available in the form of approximate (soft) logical rules, it is
not clear how to transfer it to a PLM in order to improve its performance for
deductive reasoning tasks. Here, we aim to bridge this gap by teaching PLMs how
to reason with soft Horn rules. We introduce a classification task where, given
facts and soft rules, the PLM should return a prediction with a probability for
a given hypothesis. We release the first dataset for this task, and we propose
a revised loss function that enables the PLM to learn how to predict precise
probabilities for the task. Our evaluation results show that the resulting
fine-tuned models achieve very high performance, even on logical rules that
were unseen at training. Moreover, we demonstrate that logical notions
expressed by the rules are transferred to the fine-tuned model, yielding
state-of-the-art results on external datasets.

本研究通过教授 PLMs 如何使用软 Horn 规则来提高其推理任务的性能，并通过一个分类任务和损失函数来辅助训练，从而实现了高精度的推理。