Large language models (LLMs) can "lie", which we define as outputting false
statements despite "knowing" the truth in a demonstrable sense. LLMs might
"lie", for example, when instructed to output misinformation. Here, we develop
a simple lie detector that requires neither access to the LLM's activations
(black-box) nor ground-truth knowledge of the fact in question. The detector
works by asking a predefined set of unrelated follow-up questions after a
suspected lie, and feeding the LLM's yes/no answers into a logistic regression
classifier. Despite its simplicity, this lie detector is highly accurate and
surprisingly general. When trained on examples from a single setting --
prompting GPT-3.5 to lie about factual questions -- the detector generalises
out-of-distribution to (1) other LLM architectures, (2) LLMs fine-tuned to lie,
(3) sycophantic lies, and (4) lies emerging in real-life scenarios such as
sales. These results indicate that LLMs have distinctive lie-related
behavioural patterns, consistent across architectures and contexts, which could
enable general-purpose lie detection.

通过提出一个简单的黑盒检测器，在事实问题下，只需通过预定义的一系列无关联的后续问题以及将大型语言模型的否定 / 肯定回答输入到逻辑回归分类器中，我们发现大型语言模型存在特定的与说谎相关的行为模式，这表明它们可以用于实现通用目的的说谎检测。