Large language models primarily rely on inductive reasoning for decision
making. This results in unreliable decisions when applied to real-world tasks
that often present incomplete contexts and conditions. Thus, accurate
probability estimation and appropriate interpretations are required to enhance
decision-making reliability. In this paper, we propose a Bayesian inference
framework called BIRD for large language models. BIRD provides controllable and
interpretable probability estimation for model decisions, based on abductive
factors, LLM entailment, as well as learnable deductive Bayesian modeling.
Experiments show that BIRD produces probability estimations that align with
human judgments over 65% of the time using open-sourced Llama models,
outperforming the state-of-the-art GPT-4 by 35%. We also show that BIRD can be
directly used for trustworthy decision making on many real-world applications.

本文提出了一个名为 BIRD 的贝叶斯推理框架，该框架针对大型语言模型提供了可控和可解释的概率估计，通过加入反推因素、LLM 蕴含和可学习的推导贝叶斯建模。实验表明，使用开源的 Llama 模型，BIRD 的概率估计与人类判断的一致率达到了 65%，比最先进的 GPT-4 提高了 35%。同时，我们还展示了 BIRD 可以直接用于许多真实世界应用的可靠决策。