Large Language Models (LLMs) have demonstrated remarkable capabilities for
reinforcement learning (RL) models, such as planning and reasoning
capabilities. However, the problems of LLMs and RL model collaboration still
need to be solved. In this study, we employ a teacher-student learning
framework to tackle these problems, specifically by offering feedback for LLMs
using RL models and providing high-level information for RL models with LLMs in
a cooperative multi-agent setting. Within this framework, the LLM acts as a
teacher, while the RL model acts as a student. The two agents cooperatively
assist each other through a process of recursive help, such as "I help you help
I help." The LLM agent supplies abstract information to the RL agent, enabling
efficient exploration and policy improvement. In turn, the RL agent offers
feedback to the LLM agent, providing valuable, real-time information that helps
generate more useful tokens. This bi-directional feedback loop promotes
optimization, exploration, and mutual improvement for both agents, enabling
them to accomplish increasingly challenging tasks. Remarkably, we propose a
practical algorithm to address the problem and conduct empirical experiments to
evaluate the effectiveness of our method.

大型语言模型和强化学习模型合作的教师 - 学生学习框架，通过递归互助的方式，实现了抽象信息的供给和实时反馈的互利循环，从而推动大型语言模型和强化学习模型在协同多智能体环境中的优化、探索和互相改进。