LLMs can generate factually incorrect statements even when provided access to
reference documents. Such errors can be dangerous in high-stakes applications
(e.g., document-grounded QA for healthcare or finance). We present GenAudit --
a tool intended to assist fact-checking LLM responses for document-grounded
tasks. GenAudit suggests edits to the LLM response by revising or removing
claims that are not supported by the reference document, and also presents
evidence from the reference for facts that do appear to have support. We train
models to execute these tasks, and design an interactive interface to present
suggested edits and evidence to users. Comprehensive evaluation by human raters
shows that GenAudit can detect errors in 8 different LLM outputs when
summarizing documents from diverse domains. To ensure that most errors are
flagged by the system, we propose a method that can increase the error recall
while minimizing impact on precision. We will release our tool (GenAudit) and
fact-checking model for public use.

LLMs 生成错误陈述时会产生错误，我们提出了 GenAudit，这是一个旨在辅助检查 LLM 响应的工具，通过修改或删除不被参考文献支持的主张，提供参考文献中支持的事实的证据，并设计一个交互界面将建议的修改和证据呈现给用户。