Language and Vision-Language Models (LLMs/VLMs) have revolutionized the field
of AI by their ability to generate human-like text and understand images, but
ensuring their reliability is crucial. This paper aims to evaluate the ability
of LLMs (GPT4, GPT-3.5, LLaMA2, and PaLM 2) and VLMs (GPT4V and Gemini Pro
Vision) to estimate their verbalized uncertainty via prompting. We propose the
new Japanese Uncertain Scenes (JUS) dataset, aimed at testing VLM capabilities
via difficult queries and object counting, and the Net Calibration Error (NCE)
to measure direction of miscalibration. Results show that both LLMs and VLMs
have a high calibration error and are overconfident most of the time,
indicating a poor capability for uncertainty estimation. Additionally we
develop prompts for regression tasks, and we show that VLMs have poor
calibration when producing mean/standard deviation and 95% confidence
intervals.

该论文评估了语言和视觉 - 语言模型的可靠性，并提出了新的日本不确定场景数据集以及测量校准误差的方法。结果表明，语言和视觉 - 语言模型都存在较高的校准误差，并且大部分时间都表现出过度自信，说明其对不确定性估计的能力较弱。此外，研究还发展了回归任务的提示方法，并证明视觉 - 语言模型在生成均值 / 标准差和 95% 置信区间时存在校准不良。

过度自信是关键：大型语言和视觉 - 语言模型中的口头不确定性评估

Overconfidence is Key: Verbalized Uncertainty Evaluation in Large  Language and Vision-Language Models

Large Language Models (LLMs) are increasingly used as powerful tools for
several high-stakes natural language processing (NLP) applications. Recent
prompting works claim to elicit intermediate reasoning steps and key tokens
that serve as proxy explanations for LLM predictions. However, there is no
certainty whether these explanations are reliable and reflect the LLMs
behavior. In this work, we make one of the first attempts at quantifying the
uncertainty in explanations of LLMs. To this end, we propose two novel metrics
-- $\textit{Verbalized Uncertainty}$ and $\textit{Probing Uncertainty}$ -- to
quantify the uncertainty of generated explanations. While verbalized
uncertainty involves prompting the LLM to express its confidence in its
explanations, probing uncertainty leverages sample and model perturbations as a
means to quantify the uncertainty. Our empirical analysis of benchmark datasets
reveals that verbalized uncertainty is not a reliable estimate of explanation
confidence. Further, we show that the probing uncertainty estimates are
correlated with the faithfulness of an explanation, with lower uncertainty
corresponding to explanations with higher faithfulness. Our study provides
insights into the challenges and opportunities of quantifying uncertainty in
LLM explanations, contributing to the broader discussion of the trustworthiness
of foundation models.

在这项研究中，我们尝试量化大型语言模型（LLM）解释的不确定性。为此，我们提出了两个新的度量标准 ——“口头化不确定性” 和 “探测不确定性”，用于量化生成解释的不确定性。我们的实证分析揭示了口头化不确定性不是可靠的解释置信度的估计，而探测不确定性的估计与解释的忠实度相关，较低的不确定性对应于较高的忠实度。这项研究为量化 LLM 解释的不确定性带来了洞察，有助于更广泛地探讨基础模型的可靠性。