Recent studies have shown that Large Language Models (LLMs) can be utilized
for solving complex sequential decision-making tasks by providing high-level
instructions. However, LLM-based agents face limitations in real-time dynamic
environments due to their lack of specialization in solving specific target
problems. Moreover, the deployment of such LLM-based agents is both costly and
time-consuming in practical scenarios. In this paper, we introduce a novel
framework that addresses these challenges by training a smaller scale
specialized student agent using instructions from an LLM-based teacher agent.
By leveraging guided actions provided by the teachers, the prior knowledge of
the LLM is distilled into the local student model. Consequently, the student
agent can be trained with significantly less data. Furthermore, subsequent
training with environment feedback empowers the student agents to surpass the
capabilities of their teachers. We conducted experiments on three challenging
MiniGrid environments to evaluate the effectiveness of our framework. The
results demonstrate that our approach enhances sample efficiency and achieves
superior performance compared to baseline methods.

通过使用大规模语言模型，我们提出了一种新的框架，通过从语言模型教师代理接收指导行为，训练一个小规模专用的学生代理。通过将语言模型的先验知识融入到本地学生模型中，该学生代理可以用较少的数据进行训练，并通过环境反馈进一步提升其能力。实验结果表明，我们的方法提高了样本效率，并实现了比基准方法更卓越的性能。

大型语言模型作为训练强化学习智能体的良好策略导师

Large Language Model is a Good Policy Teacher for Training Reinforcement  Learning Agents

With the rise in popularity of video-based social media, new categories of
videos are constantly being generated, creating an urgent need for robust
incremental learning techniques for video understanding. One of the biggest
challenges in this task is catastrophic forgetting, where the network tends to
forget previously learned data while learning new categories. To overcome this
issue, knowledge distillation is a widely used technique for rehearsal-based
video incremental learning that involves transferring important information on
similarities among different categories to enhance the student model.
Therefore, it is preferable to have a strong teacher model to guide the
students. However, the limited performance of the network itself and the
occurrence of catastrophic forgetting can result in the teacher network making
inaccurate predictions for some memory exemplars, ultimately limiting the
student network's performance. Based on these observations, we propose a
teacher agent capable of generating stable and accurate soft labels to replace
the output of the teacher model. This method circumvents the problem of
knowledge misleading caused by inaccurate predictions of the teacher model and
avoids the computational overhead of loading the teacher model for knowledge
distillation. Extensive experiments demonstrate the advantages of our method,
yielding significant performance improvements while utilizing only half the
resolution of video clips in the incremental phases as input compared to recent
state-of-the-art methods. Moreover, our method surpasses the performance of
joint training when employing four times the number of samples in episodic
memory.

本文旨在探究基于视频的社交媒体的应用，提出了一种新的增量学习技术 —— 知识蒸馏。该技术通过引入稳定准确的软标签来取代教师模型的输出，解决了教师模型预测不准确的问题和计算过载问题。经过大量实验，该方法在输入一半的视频分辨率下，已经可以超越最新的基准方法。