Fact-checking is an essential tool to mitigate the spread of misinformation
and disinformation. We introduce the task of fact-checking in dialogue, which
is a relatively unexplored area. We construct DialFact, a testing benchmark
dataset of 22,245 annotated conversational claims, paired with pieces of
evidence from Wikipedia. There are three sub-tasks in DialFact: 1) Verifiable
claim detection task distinguishes whether a response carries verifiable
factual information; 2) Evidence retrieval task retrieves the most relevant
Wikipedia snippets as evidence; 3) Claim verification task predicts a dialogue
response to be supported, refuted, or not enough information. We found that
existing fact-checking models trained on non-dialogue data like FEVER fail to
perform well on our task, and thus, we propose a simple yet data-efficient
solution to effectively improve fact-checking performance in dialogue. We point
out unique challenges in DialFact such as handling the colloquialisms,
coreferences and retrieval ambiguities in the error analysis to shed light on
future research in this direction.

通过构建一个名为 DialFact 的测试基准数据集，该数据集包含了 22,245 个带注释的对话声明和来自维基百科的证据片段，并提出了在对话中事实核查的三个子任务：可验证的声明检测、证据检索和索取的内容足够详细的陈述是否被支持。针对 DialFact 的独特挑战，本文提出了一种简单而数据效率高的解决方案，以有效提高对话中的事实核查性能。