Recent advancements in reasoning abilities of Large Language Models (LLM) has
promoted their usage in problems that require high-level planning for robots
and artificial agents. However, current techniques that utilize LLMs for such
planning tasks make certain key assumptions such as, access to datasets that
permit finetuning, meticulously engineered prompts that only provide relevant
and essential information to the LLM, and most importantly, a deterministic
approach to allow execution of the LLM responses either in the form of existing
policies or plan operators. In this work, we propose LgTS (LLM-guided
Teacher-Student learning), a novel approach that explores the planning
abilities of LLMs to provide a graphical representation of the sub-goals to a
reinforcement learning (RL) agent that does not have access to the transition
dynamics of the environment. The RL agent uses Teacher-Student learning
algorithm to learn a set of successful policies for reaching the goal state
from the start state while simultaneously minimizing the number of
environmental interactions. Unlike previous methods that utilize LLMs, our
approach does not assume access to a propreitary or a fine-tuned LLM, nor does
it require pre-trained policies that achieve the sub-goals proposed by the LLM.
Through experiments on a gridworld based DoorKey domain and a search-and-rescue
inspired domain, we show that generating a graphical structure of sub-goals
helps in learning policies for the LLM proposed sub-goals and the
Teacher-Student learning algorithm minimizes the number of environment
interactions when the transition dynamics are unknown.

本研究提出了一种新方法，利用大型语言模型（LLM）的规划能力，为没有环境转换动力学访问权限的强化学习代理提供子目标的图形表示，同时最小化环境交互次数。