Step-by-step decision planning with large language models (LLMs) is gaining
attention in AI agent development. This paper focuses on decision planning with
uncertainty estimation to address the hallucination problem in language models.
Existing approaches are either white-box or computationally demanding, limiting
use of black-box proprietary LLMs within budgets. The paper's first
contribution is a non-parametric uncertainty quantification method for LLMs,
efficiently estimating point-wise dependencies between input-decision on the
fly with a single inference, without access to token logits. This estimator
informs the statistical interpretation of decision trustworthiness. The second
contribution outlines a systematic design for a decision-making agent,
generating actions like ``turn on the bathroom light'' based on user prompts
such as ``take a bath''. Users will be asked to provide preferences when more
than one action has high estimated point-wise dependencies. In conclusion, our
uncertainty estimation and decision-making agent design offer a cost-efficient
approach for AI agent development.

该论文关注使用大型语言模型进行决策规划，并通过不确定性估计来解决语言模型中的幻觉问题，提出了非参数化的不确定性量化方法和决策代理设计的系统化方法，为人工智能代理的开发提供了一种高效的成本有效途径。