Large Language Models (LLMs) have demonstrated impressive capabilities across
various domains, prompting a surge in their practical applications. However,
concerns have arisen regarding the trustworthiness of LLMs outputs,
particularly in closed-book question-answering tasks, where non-experts may
struggle to identify inaccuracies due to the absence of contextual or ground
truth information. This paper introduces TrustScore, a framework based on the
concept of Behavioral Consistency, which evaluates whether an LLMs response
aligns with its intrinsic knowledge. Additionally, TrustScore can seamlessly
integrate with fact-checking methods, which assesses alignment with external
knowledge sources. The experimental results show that TrustScore achieves
strong correlations with human judgments, surpassing existing reference-free
metrics, and achieving results on par with reference-based metrics.

本研究提出了基于行为一致性概念的 TrustScore 框架，用于评估大型语言模型（LLMs）的响应与其内在知识的一致性，同时能够与事实核实方法无缝集成，实现与人类判断强相关性的结果。