Decision making via sequence modeling aims to mimic the success of language
models, where actions taken by an embodied agent are modeled as tokens to
predict. Despite their promising performance, it remains unclear if embodied
sequence modeling leads to the emergence of internal representations that
represent the environmental state information. A model that lacks abstract
state representations would be liable to make decisions based on surface
statistics which fail to generalize. We take the BabyAI environment, a grid
world in which language-conditioned navigation tasks are performed, and build a
sequence modeling Transformer, which takes a language instruction, a sequence
of actions, and environmental observations as its inputs. In order to
investigate the emergence of abstract state representations, we design a
"blindfolded" navigation task, where only the initial environmental layout, the
language instruction, and the action sequence to complete the task are
available for training. Our probing results show that intermediate
environmental layouts can be reasonably reconstructed from the internal
activations of a trained model, and that language instructions play a role in
the reconstruction accuracy. Our results suggest that many key features of
state representations can emerge via embodied sequence modeling, supporting an
optimistic outlook for applications of sequence modeling objectives to more
complex embodied decision-making domains.

通过序列建模决策，目标是模仿语言模型的成功，其中通过建模代理行为来预测。尽管性能可观，但尚不清楚是否通过序列建模产生了代表环境状态信息的内部表示。本研究利用 BabyAI 环境（一个基于语言的导航任务的格子世界），构建了一个以语言指令、行动序列和环境观测作为输入的序列建模 Transformer，并设计了一个 “蒙眼” 导航任务，用于研究抽象状态表示的出现。实验结果显示，通过已训练模型的内部激活，可以合理地重构中间环境布局，并且语言指令对重构准确性起作用。研究结果表明，通过序列建模，可以产生状态表示的许多关键特征，支持将序列建模应用于更复杂的实际决策领域。