Detecting non-factual content is a longstanding goal to increase the
trustworthiness of large language models (LLMs) generations. Current factuality
probes, trained using humanannotated labels, exhibit limited transferability to
out-of-distribution content, while online selfconsistency checking imposes
extensive computation burden due to the necessity of generating multiple
outputs. This paper proposes PINOSE, which trains a probing model on offline
self-consistency checking results, thereby circumventing the need for
human-annotated data and achieving transferability across diverse data
distributions. As the consistency check process is offline, PINOSE reduces the
computational burden of generating multiple responses by online consistency
verification. Additionally, it examines various aspects of internal states
prior to response decoding, contributing to more effective detection of factual
inaccuracies. Experiment results on both factuality detection and question
answering benchmarks show that PINOSE achieves surpassing results than existing
factuality detection methods. Our code and datasets are publicly available on
this anonymized repository.

使用 PINOSE 模型进行离线自洽检查有效降低了在线一致性检验的计算负担，并在事实检测和问答基准测试中取得了优于现有方法的结果。

通过离线一致性检查的探头训练实现可转移和高效的非事实内容检测

Transferable and Efficient Non-Factual Content Detection via Probe  Training with Offline Consistency Checking

Factuality can play an important role when automatically processing clinical
text, as it makes a difference if particular symptoms are explicitly not
present, possibly present, not mentioned, or affirmed. In most cases, a
sufficient number of examples is necessary to handle such phenomena in a
supervised machine learning setting. However, as clinical text might contain
sensitive information, data cannot be easily shared. In the context of
factuality detection, this work presents a simple solution using machine
translation to translate English data to German to train a transformer-based
factuality detection model.

使用机器翻译将英文数据翻译为德文，以训练基于变压器的事实检测模型，以解决临床文本中的事实性问题。