Language models (LMs) have demonstrated remarkable capabilities across a wide
range of tasks in various domains. Despite their impressive performance, the
reliability of their output is concerning and questionable regarding the demand
for AI safety. Assessing the confidence of LM predictions and calibrating them
across different tasks with the aim to align LM confidence with accuracy can
help mitigate risks and enable LMs to make better decisions. There have been
various works in this respect, but there has been no comprehensive overview of
this important research area. The present survey aims to bridge this gap. In
particular, we discuss methods and techniques for LM confidence estimation and
calibration, encompassing different LMs and various tasks. We further outline
the challenges of estimating the confidence for large language models and we
suggest some promising directions for future work.

评估语言模型预测的可靠性和置信度以及解决其与 AI 安全需求的关系是一项重要研究领域，本文综述了语言模型置信度估计和校准的方法、技术和挑战，并提出了未来研究的方向。

语言模型置信度评估与校准调查

A Survey of Language Model Confidence Estimation and Calibration

Much of the knowledge encoded in transformer language models (LMs) may be
expressed in terms of relations: relations between words and their synonyms,
entities and their attributes, etc. We show that, for a subset of relations,
this computation is well-approximated by a single linear transformation on the
subject representation. Linear relation representations may be obtained by
constructing a first-order approximation to the LM from a single prompt, and
they exist for a variety of factual, commonsense, and linguistic relations.
However, we also identify many cases in which LM predictions capture relational
knowledge accurately, but this knowledge is not linearly encoded in their
representations. Our results thus reveal a simple, interpretable, but
heterogeneously deployed knowledge representation strategy in transformer LMs.

基于 transformer 语言模型的研究中，发现存在一种简单而可解释的、但在各种情况下应用不一的知识表示策略，其主要通过关系、线性变换以及预测表达出来。