Uncertainty learning and quantification of models are crucial tasks to
enhance the trustworthiness of the models. Importantly, the recent surge of
generative language models (GLMs) emphasizes the need for reliable uncertainty
quantification due to the concerns on generating hallucinated facts. In this
paper, we propose to learn neural prediction set models that comes with the
probably approximately correct (PAC) guarantee for quantifying the uncertainty
of GLMs. Unlike existing prediction set models, which are parameterized by a
scalar value, we propose to parameterize prediction sets via neural networks,
which achieves more precise uncertainty quantification but still satisfies the
PAC guarantee. We demonstrate the efficacy of our method on four types of
language datasets and six types of models by showing that our method improves
the quantified uncertainty by $63\%$ on average, compared to a standard
baseline method.

通过神经预测集模型以更准确地量化不确定性，本文提出的方法在四种语言数据集和六种模型上表现出 63% 的平均不确定性改进，相对于标准基准方法来说。