Planning, as the core module of agents, is crucial in various fields such as
embodied agents, web navigation, and tool using. With the development of large
language models (LLMs), some researchers treat large language models as
intelligent agents to stimulate and evaluate their planning capabilities.
However, the planning mechanism is still unclear. In this work, we focus on
exploring the look-ahead planning mechanism in large language models from the
perspectives of information flow and internal representations. First, we study
how planning is done internally by analyzing the multi-layer perception (MLP)
and multi-head self-attention (MHSA) components at the last token. We find that
the output of MHSA in the middle layers at the last token can directly decode
the decision to some extent. Based on this discovery, we further trace the
source of MHSA by information flow, and we reveal that MHSA mainly extracts
information from spans of the goal states and recent steps. According to
information flow, we continue to study what information is encoded within it.
Specifically, we explore whether future decisions have been encoded in advance
in the representation of flow. We demonstrate that the middle and upper layers
encode a few short-term future decisions to some extent when planning is
successful. Overall, our research analyzes the look-ahead planning mechanisms
of LLMs, facilitating future research on LLMs performing planning tasks.

我们研究了大型语言模型中的预测规划机制，通过分析信息流和内部表示的角度，我们发现多层感知器和多头自注意力组件在最后一个标记上的输出可以直接解码决策，在信息流方面，我们揭示了多头自注意力主要提取目标状态和最近步骤的范围内的信息，我们进一步探索了这些信息的编码方式，并展示了中间和上层在成功规划时在一定程度上编码了一些短期未来决策，这项研究有助于未来在大型语言模型执行规划任务方面的研究。