This paper focuses on extending the success of large language models (LLMs)
to sequential decision making. Existing efforts either (i) re-train or finetune
LLMs for decision making, or (ii) design prompts for pretrained LLMs. The
former approach suffers from the computational burden of gradient updates, and
the latter approach does not show promising results. In this paper, we propose
a new approach that leverages online model selection algorithms to efficiently
incorporate LLMs agents into sequential decision making. Statistically, our
approach significantly outperforms both traditional decision making algorithms
and vanilla LLM agents. Computationally, our approach avoids the need for
expensive gradient updates of LLMs, and throughout the decision making process,
it requires only a small number of LLM calls. We conduct extensive experiments
to verify the effectiveness of our proposed approach. As an example, on a
large-scale Amazon dataset, our approach achieves more than a $6$x performance
gain over baselines while calling LLMs in only $1.5$\% of the time steps.

本论文提出了一种新的方法，利用在线模型选择算法在序列决策中高效地整合 LLM 代理，统计上显著优于传统决策算法和普通 LLM 代理，计算上避免了 LLM 梯度更新的高昂代价，并且在整个决策过程中只需要少量 LLM 调用。