Commonsense reasoning is an appealing topic in natural language processing
(NLP) as it plays a fundamental role in supporting the human-like actions of
NLP systems. With large-scale language models as the backbone, unsupervised
pre-training on numerous corpora shows the potential to capture commonsense
knowledge. Current pre-trained language model (PLM)-based reasoning follows the
traditional practice using perplexity metric. However, commonsense reasoning is
more than existing probability evaluation, which is biased by word frequency.
This paper reconsiders the nature of commonsense reasoning and proposes a novel
commonsense reasoning metric, Non-Replacement Confidence (NRC). In detail, it
works on PLMs according to the Replaced Token Detection (RTD) pre-training
objective in ELECTRA, in which the corruption detection objective reflects the
confidence on contextual integrity that is more relevant to commonsense
reasoning than existing probability. Our proposed novel method boosts zero-shot
performance on two commonsense reasoning benchmark datasets and further seven
commonsense question-answering datasets. Our analysis shows that pre-endowed
commonsense knowledge, especially for RTD-based PLMs, is essential in
downstream reasoning.

本文提出一种新的通识推理度量标准 ——Non-Replacement Confidence (NRC)，通过 Replaced Token Detection (RTD) 预训练目标在 PLMs 上进行推理，提高了两个通识推理基准数据集和七个通识问题回答数据集的零 - shot 表现。