Autonomous fact-checking, using machine learning to verify claims, has grown
vital as misinformation spreads beyond human fact-checking capacity. Large
Language Models (LLMs) like GPT-4 are increasingly trusted to verify
information and write academic papers, lawsuits, and news articles, emphasizing
their role in discerning truth from falsehood and the importance of being able
to verify their outputs. Here, we evaluate the use of LLM agents in
fact-checking by having them phrase queries, retrieve contextual data, and make
decisions. Importantly, in our framework, agents explain their reasoning and
cite the relevant sources from the retrieved context. Our results show the
enhanced prowess of LLMs when equipped with contextual information. GPT-4
outperforms GPT-3, but accuracy varies based on query language and claim
veracity. While LLMs show promise in fact-checking, caution is essential due to
inconsistent accuracy. Our investigation calls for further research, fostering
a deeper comprehension of when agents succeed and when they fail.

自动事实核查（使用机器学习来验证主张）已经变得至关重要，因为虚假信息已经超出了人类事实核查的能力范围。大型语言模型（LLMs），如 GPT-4，越来越受人们的信任，可以验证信息并撰写学术论文、诉讼文件和新闻文章，强调了它们在分辨真假和能够验证其输出的重要性。在这里，我们通过让 LLM 代理人表达查询、检索上下文数据和做出决策来评估 LLMs 在事实核查中的使用。重要的是，在我们的框架中，代理人解释他们的推理并引用检索到的相关来源。我们的结果显示，在配备上下文信息的情况下，LLMs 表现出更强大的能力。GPT-4 优于 GPT-3，但准确性取决于查询语言和主张的真实性。虽然 LLMs 在事实核查方面显示出前景，但仍需要谨慎使用，因为准确性不一致。我们的调研呼吁进一步研究，以更深入地了解代理人何时成功以及何时失败。