Pre-trained language models (PLMs) have shown unprecedented potential in
various fields, especially as the backbones for question-answering (QA)
systems. However, they tend to be easily deceived by tricky questions such as
"How many eyes does the sun have?". Such frailties of PLMs often allude to the
lack of knowledge within them. In this paper, we find that the PLMs already
possess the knowledge required to rebut such questions, and the key is how to
activate the knowledge. To systematize this observation, we investigate the
PLMs' responses to one kind of tricky questions, i.e., the false premises
questions (FPQs). We annotate a FalseQA dataset containing 2365 human-written
FPQs, with the corresponding explanations for the false premises and the
revised true premise questions. Using FalseQA, we discover that PLMs are
capable of discriminating FPQs by fine-tuning on moderate numbers (e.g., 256)
of examples. PLMs also generate reasonable explanations for the false premise,
which serve as rebuttals. Further replaying a few general questions during
training allows PLMs to excel on FPQs and general questions simultaneously. Our
work suggests that once the rebuttal ability is stimulated, knowledge inside
the PLMs can be effectively utilized to handle FPQs, which incentivizes the
research on PLM-based QA systems.

本文发现预训练语言模型内部已经拥有反驳谣言的所需知识，只需要激发其反驳能力，通过 fine-tuning 可以在少量示例（例如 256 个）的情况下区分虚假前提问题，并生成合理的反驳解释。