As the use of Large Language Models (LLMs) becomes more widespread,
understanding their self-evaluation of confidence in generated responses
becomes increasingly important as it is integral to the reliability of the
output of these models. We introduce the concept of Confidence-Probability
Alignment, that connects an LLM's internal confidence, quantified by token
probabilities, to the confidence conveyed in the model's response when
explicitly asked about its certainty. Using various datasets and prompting
techniques that encourage model introspection, we probe the alignment between
models' internal and expressed confidence. These techniques encompass using
structured evaluation scales to rate confidence, including answer options when
prompting, and eliciting the model's confidence level for outputs it does not
recognize as its own. Notably, among the models analyzed, OpenAI's GPT-4 showed
the strongest confidence-probability alignment, with an average Spearman's
$\hat{\rho}$ of 0.42, across a wide range of tasks. Our work contributes to the
ongoing efforts to facilitate risk assessment in the application of LLMs and to
further our understanding of model trustworthiness.

通过使用各种数据集和提示技术，本研究探索了大型语言模型（LLMs）内部信心与其对外表达的一致性，其中 OpenAI 的 GPT-4 表现出最强的信心 - 概率对齐性，并有助于评估 LLMs 的风险和提高对模型可信度的认识。