The increased use of large language models (LLMs) across a variety of
real-world applications calls for mechanisms to verify the factual accuracy of
their outputs. Difficulties lie in assessing the factuality of free-form
responses in open domains. Also, different papers use disparate evaluation
benchmarks and measurements, which renders them hard to compare and hampers
future progress. To mitigate these issues, we propose OpenFactCheck, a unified
factuality evaluation framework for LLMs. OpenFactCheck consists of three
modules: (i) CUSTCHECKER allows users to easily customize an automatic
fact-checker and verify the factual correctness of documents and claims, (ii)
LLMEVAL, a unified evaluation framework assesses LLM's factuality ability from
various perspectives fairly, and (iii) CHECKEREVAL is an extensible solution
for gauging the reliability of automatic fact-checkers' verification results
using human-annotated datasets. OpenFactCheck is publicly released at
this https URL

基于大型语言模型（LLMs）的事实性评估框架 OpenFactCheck，包括自定义自动事实检查器 CUSTCHECKER、多角度评估 LMM 事实性的统一评估框架 LLMEVAL 和使用人工标注数据集评估自动事实检查器验证结果可靠性的 CHECKEREVAL。