Existing knowledge-enhanced methods have achieved remarkable results in
certain QA tasks via obtaining diverse knowledge from different knowledge
bases. However, limited by the properties of retrieved knowledge, they still
have trouble benefiting from both the knowledge relevance and distinguishment
simultaneously. To address the challenge, we propose CPACE, a Concept-centric
Prompt-bAsed Contrastive Explanation Generation model, which aims to convert
obtained symbolic knowledge into a contrastive explanation for better
distinguishing the differences among given candidates. Firstly, following
previous works, we retrieve different types of symbolic knowledge with a
concept-centric knowledge extraction module. After that, we generate
corresponding contrastive explanations using acquired symbolic knowledge and
explanation prompts as guidance for better modeling the knowledge
distinguishment and interpretability. Finally, we regard the generated
contrastive explanation as external knowledge for downstream task enhancement.
We conduct a series of experiments on three widely-used question-answering
datasets: CSQA, QASC, and OBQA. Experimental results demonstrate that with the
help of generated contrastive explanation, our CPACE model achieves new SOTA on
CSQA (89.8% on the testing set, 0.9% higher than human performance), and gains
impressive improvement on QASC and OBQA (4.2% and 3.5%, respectively).

使用 CPACE 模型和符号知识和解释提示来生成对比性解释，从而帮助区分问题答案，提高知识增强型方法在问答任务中的表现。

提出撰写对比解释作为常识问题回答知识的先验鉴别方法

Distinguish Before Answer: Generating Contrastive Explanation as  Knowledge for Commonsense Question Answering

Pre-trained language models (PTLM) have achieved impressive results in a
range of natural language understanding (NLU) and generation (NLG) tasks.
However, current pre-training objectives such as masked token prediction (for
BERT-style PTLMs) and masked span infilling (for T5-style PTLMs) do not
explicitly model the relational commonsense knowledge about everyday concepts,
which is crucial to many downstream tasks that need common sense to understand
or generate. To augment PTLMs with concept-centric commonsense knowledge, in
this paper, we propose both generative and contrastive objectives for learning
common sense from the text, and use them as intermediate self-supervised
learning tasks for incrementally pre-training PTLMs (before task-specific
fine-tuning on downstream datasets). Furthermore, we develop a joint
pre-training framework to unify generative and contrastive objectives so that
they can mutually reinforce each other. Extensive experimental results show
that our method, concept-aware language model (CALM), can pack more commonsense
knowledge into the parameters of a pre-trained text-to-text transformer without
relying on external knowledge graphs, yielding better performance on both NLU
and NLG tasks. We show that while only incrementally pre-trained on a
relatively small corpus for a few steps, CALM outperforms baseline methods by a
consistent margin and even comparable with some larger PTLMs, which suggests
that CALM can serve as a general, plug-and-play method for improving the
commonsense reasoning ability of a PTLM.

本文提出概念感知的语言模型 (CALM)，通过增强自监督学习任务以从文本中学习常识。CALM 可以在不依赖外部知识图谱的情况下将更多的常识知识整合到预先训练的文本转文本变压器的参数中，从而在自然语言理解和生成任务中实现更好的性能。