The formidable capacity for zero- or few-shot decision-making in language
agents encourages us to pose a compelling question: Can language agents be
alternatives to PPO agents in traditional sequential decision-making tasks? To
investigate this, we first take environments collected in OpenAI Gym as our
testbeds and ground them to textual environments that construct the TextGym
simulator. This allows for straightforward and efficient comparisons between
PPO agents and language agents, given the widespread adoption of OpenAI Gym. To
ensure a fair and effective benchmarking, we introduce $5$ levels of scenario
for accurate domain-knowledge controlling and a unified RL-inspired framework
for language agents. Additionally, we propose an innovative
explore-exploit-guided language (EXE) agent to solve tasks within TextGym.
Through numerical experiments and ablation studies, we extract valuable
insights into the decision-making capabilities of language agents and make a
preliminary evaluation of their potential to be alternatives to PPO in
classical sequential decision-making problems. This paper sheds light on the
performance of language agents and paves the way for future research in this
exciting domain. Our code is publicly available
at~https://github.com/mail-ecnu/Text-Gym-Agents.

语言代理是否能成为传统序列决策任务中 PPO 代理的替代品？本研究通过在 OpenAI Gym 中构建 TextGym 模拟器，对 PPO 代理和语言代理进行有效的比较，提出了基于统一强化学习框架和五个精确的领域知识控制场景水平的 RL-inspired 语言探索利用（EXE）代理来解决任务。通过数值实验和消融研究，我们对语言代理的决策能力进行了有价值的探索，并进行了初步评估其在经典序列决策问题中成为 PPO 代理替代品的潜力。本研究为语言代理的性能提供了新的认识，并为该领域的未来研究铺平了道路。