Uncertainty quantification approaches have been more critical in large
language models (LLMs), particularly high-risk applications requiring reliable
outputs. However, traditional methods for uncertainty quantification, such as
probabilistic models and ensemble techniques, face challenges when applied to
the complex and high-dimensional nature of LLM-generated outputs. This study
proposes a novel geometric approach to uncertainty quantification using convex
hull analysis. The proposed method leverages the spatial properties of response
embeddings to measure the dispersion and variability of model outputs. The
prompts are categorized into three types, i.e., `easy', `moderate', and
`confusing', to generate multiple responses using different LLMs at varying
temperature settings. The responses are transformed into high-dimensional
embeddings via a BERT model and subsequently projected into a two-dimensional
space using Principal Component Analysis (PCA). The Density-Based Spatial
Clustering of Applications with Noise (DBSCAN) algorithm is utilized to cluster
the embeddings and compute the convex hull for each selected cluster. The
experimental results indicate that the uncertainty of the model for LLMs
depends on the prompt complexity, the model, and the temperature setting.

该研究提出了一种新颖的几何方法来评估大语言模型（LLM）的不确定性，利用凸包分析来度量模型输出的离散度和可变性，该方法通过将回答转化为高维嵌入，并使用主成分分析（PCA）将其投影到二维空间。实验结果表明，LLMs 模型的不确定性取决于提示的复杂性、模型和温度设置。