The surprisingly likely criterion in the seminal work of Prelec (the Bayesian
Truth Serum) guarantees truthfulness in a game-theoretic multi-agent setting,
by rewarding rational agents to maximise the expected information gain with
their answers w.r.t. their probabilistic beliefs. We investigate the relevance
of a similar criterion for responses of LLMs. We hypothesize that if the
surprisingly likely criterion works in LLMs, under certain conditions, the
responses that maximize the reward under this criterion should be more accurate
than the responses that only maximize the posterior probability. Using
benchmarks including the TruthfulQA benchmark and using openly available LLMs:
GPT-2 and LLaMA-2, we show that the method indeed improves the accuracy
significantly (for example, upto 24 percentage points aggregate improvement on
TruthfulQA and upto 70 percentage points improvement on individual categories
of questions).

在这篇研究中，我们调查了类似于 Bayesian Truth Serum 的标准对 LLMs 的回答的相关性，并假设在一定条件下，根据这一标准最大化奖励的回答应该比仅最大化后验概率的回答更准确。通过使用包括 TruthfulQA 基准和开放可用的 LLMs（如 GPT-2 和 LLaMA-2）的基准实验证明该方法确实显著提高了准确性（例如，在 TruthfulQA 上的总体改进高达 24 个百分点，在各个问题类别上的改进高达 70 个百分点）。