This paper considers the challenges that Large Language Models (LLMs) face
when reasoning over text that includes information involving uncertainty
explicitly quantified via probability values. This type of reasoning is
relevant to a variety of contexts ranging from everyday conversations to
medical decision-making. Despite improvements in the mathematical reasoning
capabilities of LLMs, they still exhibit significant difficulties when it comes
to probabilistic reasoning. To deal with this problem, we first introduce the
Bayesian Linguistic Inference Dataset (BLInD), a new dataset specifically
designed to test the probabilistic reasoning capabilities of LLMs. We then
leverage this new dataset to thoroughly illustrate the specific limitations of
LLMs for tasks involving probabilistic reasoning and present several strategies
that map the problem to different formal representations, including Python
code, probabilistic inference algorithms, and probabilistic logical
programming. We conclude by providing an evaluation of our methods on BLInD and
on an adaptation of a causal reasoning question-answering dataset, which
further shows their practical effectiveness.

本研究考虑了大型语言模型（LLMs）在推理涉及概率值明确量化的文本时面临的挑战。通过引入贝叶斯语言推理数据集（BLInD），我们详细说明了 LLMs 在涉及概率推理的任务中的限制，并提出了几种将问题映射到不同形式表示的策略。我们通过在 BLInD 上进行评估以及对因果推理问答数据集的适应性改编等实验证明了这些方法的实际效果。