Reinforcement Learning (RL) plays an important role in the robotic
manipulation domain since it allows self-learning from trial-and-error
interactions with the environment. Still, sample efficiency and reward
specification seriously limit its potential. One possible solution involves
learning from expert guidance. However, obtaining a human expert is impractical
due to the high cost of supervising an RL agent, and developing an automatic
supervisor is a challenging endeavor. Large Language Models (LLMs) demonstrate
remarkable abilities to provide human-like feedback on user inputs in natural
language. Nevertheless, they are not designed to directly control low-level
robotic motions, as their pretraining is based on vast internet data rather
than specific robotics data. In this paper, we introduce the Lafite-RL
(Language agent feedback interactive Reinforcement Learning) framework, which
enables RL agents to learn robotic tasks efficiently by taking advantage of
LLMs' timely feedback. Our experiments conducted on RLBench tasks illustrate
that, with simple prompt design in natural language, the Lafite-RL agent
exhibits improved learning capabilities when guided by an LLM. It outperforms
the baseline in terms of both learning efficiency and success rate,
underscoring the efficacy of the rewards provided by an LLM.

通过利用大型语言模型的及时反馈，Lafite-RL（语言代理反馈互动式强化学习）框架使强化学习智能体能够有效地学习机器人任务，实验结果表明，Lafite-RL 智能体在自然语言的简单提示设计下，通过大型语言模型的引导在学习效率和成功率方面优于基准模型，凸显了大型语言模型所提供的奖励的功效。