Recognizing if LLM output can be grounded in evidence is central to many
tasks in NLP: retrieval-augmented generation, summarization, document-grounded
dialogue, and more. Current approaches to this kind of "fact-checking" are
based on verifying each piece of a model generation against potential evidence
using an LLM. However, this process can be very computationally expensive,
requiring many calls to LLMs to check a single response. In this work, we show
how to build small models that have GPT-4-level performance but for 400x lower
cost. We do this by constructing synthetic training data with GPT-4, which
involves creating realistic yet challenging instances of factual errors via a
structured generation procedure. Training on this data teaches models to check
each fact in the claim and recognize synthesis of information across sentences.
For evaluation, we unify pre-existing datasets into a benchmark LLM-AggreFact,
collected from recent work on fact-checking and grounding LLM generations. Our
best system MiniCheck-FT5 (770M parameters) outperforms all systems of
comparable size and reaches GPT-4 accuracy. We release LLM-AggreFact, code for
data synthesis, and models.

通过构建合成训练数据，我们展示了如何构建具有 GPT-4 水平性能但成本降低 400 倍的小型模型，通过检查声明中的每个事实并识别跨句子的信息合成来进行验证，并发布了 LLM-AggreFact 数据集和模型。