While large language models (LMs) have shown remarkable capabilities across
numerous tasks, they often struggle with simple reasoning and planning in
physical environments, such as understanding object permanence or planning
household activities. The limitation arises from the fact that LMs are trained
only on written text and miss essential embodied knowledge and skills. In this
paper, we propose a new paradigm of enhancing LMs by finetuning them with world
models, to gain diverse embodied knowledge while retaining their general
language capabilities. Our approach deploys an embodied agent in a world model,
particularly a simulator of the physical world (VirtualHome), and acquires a
diverse set of embodied experiences through both goal-oriented planning and
random exploration. These experiences are then used to finetune LMs to teach
diverse abilities of reasoning and acting in the physical world, e.g., planning
and completing goals, object permanence and tracking, etc. Moreover, it is
desirable to preserve the generality of LMs during finetuning, which
facilitates generalizing the embodied knowledge across tasks rather than being
tied to specific simulations. We thus further introduce the classical elastic
weight consolidation (EWC) for selective weight updates, combined with low-rank
adapters (LoRA) for training efficiency. Extensive experiments show our
approach substantially improves base LMs on 18 downstream tasks by 64.28% on
average. In particular, the small LMs (1.3B and 6B) enhanced by our approach
match or even outperform much larger LMs (e.g., ChatGPT).

本文提出了一种通过 fine-tuning 使用世界模型使大型自然语言模型获得基于物理环境的知识和技能，以帮助解决模型在简单的推理和规划中的局限性，实验证明这种方法能够提高基本的 LM 在 18 项下游任务上的性能，尤其是 1.3B 和 6B 的小 LM。