Our society is facing rampant misinformation harming public health and trust.
To address the societal challenge, we introduce FACT-GPT, a system leveraging
Large Language Models (LLMs) to automate the claim matching stage of
fact-checking. FACT-GPT, trained on a synthetic dataset, identifies social
media content that aligns with, contradicts, or is irrelevant to previously
debunked claims. Our evaluation shows that our specialized LLMs can match the
accuracy of larger models in identifying related claims, closely mirroring
human judgment. This research provides an automated solution for efficient
claim matching, demonstrates the potential of LLMs in supporting fact-checkers,
and offers valuable resources for further research in the field.

我们提出了 FACT-GPT 这个系统，利用大型语言模型 (LLMs) 自动化事实核查中的索引匹配阶段。FACT-GPT 经训练后，可以识别与先前被揭穿的声明相符、相悖或无关的社交媒体内容。我们的评估结果表明，我们专门训练的 LLMs 在识别相关声明方面的准确性与更大型的模型相当，与人类判断非常接近。这项研究提供了一种高效的声明匹配自动化解决方案，展示了 LLMs 在支持事实核查员方面的潜力，并为该领域的进一步研究提供了宝贵的资源。

FACT-GPT: 通过与 LLMs 进行索引匹配的事实检查增强

FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMs

In today's digital era, the rapid spread of misinformation poses threats to
public well-being and societal trust. As online misinformation proliferates,
manual verification by fact checkers becomes increasingly challenging. We
introduce FACT-GPT (Fact-checking Augmentation with Claim matching
Task-oriented Generative Pre-trained Transformer), a framework designed to
automate the claim matching phase of fact-checking using Large Language Models
(LLMs). This framework identifies new social media content that either supports
or contradicts claims previously debunked by fact-checkers. Our approach
employs GPT-4 to generate a labeled dataset consisting of simulated social
media posts. This data set serves as a training ground for fine-tuning more
specialized LLMs. We evaluated FACT-GPT on an extensive dataset of social media
content related to public health. The results indicate that our fine-tuned LLMs
rival the performance of larger pre-trained LLMs in claim matching tasks,
aligning closely with human annotations. This study achieves three key
milestones: it provides an automated framework for enhanced fact-checking;
demonstrates the potential of LLMs to complement human expertise; offers public
resources, including datasets and models, to further research and applications
in the fact-checking domain.

通过使用 FACT-GPT 框架，我们介绍了一种自动化事实核查的方法，该方法利用大型语言模型 (LLMs) 的主张匹配阶段来识别新的社交媒体内容，无论是支持还是反驳之前被事实核查人员驳斥的主张。研究结果表明，我们的精细调节的 LLMs 在主张匹配任务中与更大型的预训练 LLMs 的性能相媲美，与人工标注结果密切一致。