Generative commonsense question answering (GenCQA) is a task of automatically
generating a list of answers given a question. The answer list is required to
cover all reasonable answers. This presents the considerable challenges of
producing diverse answers and ranking them properly. Incorporating a variety of
closely-related background knowledge into the encoding of questions enables the
generation of different answers. Meanwhile, learning to distinguish positive
answers from negative ones potentially enhances the probabilistic estimation of
plausibility, and accordingly, the plausibility-based ranking. Therefore, we
propose a Knowledge Enhancement and Plausibility Ranking (KEPR) approach
grounded on the Generate-Then-Rank pipeline architecture. Specifically, we
expand questions in terms of Wiktionary commonsense knowledge of keywords, and
reformulate them with normalized patterns. Dense passage retrieval is utilized
for capturing relevant knowledge, and different PLM-based (BART, GPT2 and T5)
networks are used for generating answers. On the other hand, we develop an
ELECTRA-based answer ranking model, where logistic regression is conducted
during training, with the aim of approximating different levels of plausibility
in a polar classification scenario. Extensive experiments on the benchmark
ProtoQA show that KEPR obtains substantial improvements, compared to the strong
baselines. Within the experimental models, the T5-based GenCQA with KEPR
obtains the best performance, which is up to 60.91% at the primary canonical
metric Inc@3. It outperforms the existing GenCQA models on the current
leaderboard of ProtoQA.

提出了一种基于知识增强和真实性排名的生成式常识问答方法，其中涵盖关键词的常识知识被用于对问题进行扩展并进行归一化，密集通道检索用于捕获相关知识，而不同的 PLM（BART，GPT2 和 T5）网络用于生成答案。而基于 ELECTRA 的答案排名模型则在训练期间进行逻辑回归，以近似极性分类情况下的不同可信度水平。实验结果表明该方法在 ProtoQA 基准测试上的表现优于现有方法。

KEPR: 生成通用常识问题回答的知识增强和可信度排名

KEPR: Knowledge Enhancement and Plausibility Ranking for Generative  Commonsense Question Answering

Fine-tuning of pre-trained transformer models has become the standard
approach for solving common NLP tasks. Most of the existing approaches rely on
a randomly initialized classifier on top of such networks. We argue that this
fine-tuning procedure is sub-optimal as the pre-trained model has no prior on
the specific classifier labels, while it might have already learned an
intrinsic textual representation of the task. In this paper, we introduce a new
scoring method that casts a plausibility ranking task in a full-text format and
leverages the masked language modeling head tuned during the pre-training
phase. We study commonsense reasoning tasks where the model must rank a set of
hypotheses given a premise, focusing on the COPA, Swag, HellaSwag and
CommonsenseQA datasets. By exploiting our scoring method without fine-tuning,
we are able to produce strong baselines (e.g. 80% test accuracy on COPA) that
are comparable to supervised approaches. Moreover, when fine-tuning directly on
the proposed scoring function, we show that our method provides a much more
stable training phase across random restarts (e.g $\times 10$ standard
deviation reduction on COPA test accuracy) and requires less annotated data
than the standard classifier approach to reach equivalent performances.

本研究提出一种基于预训练 transformer 模型的全文格式的可信度排名得分方法，无需微调即可产生强大的基线，可以为通识推理任务提供更加稳定和有效的训练解决方案。