With the surge in the development of large language models, embodied
intelligence has attracted increasing attention. Nevertheless, prior works on
embodied intelligence typically encode scene or historical memory in an
unimodal manner, either visual or linguistic, which complicates the alignment
of the model's action planning with embodied control. To overcome this
limitation, we introduce the Multimodal Embodied Interactive Agent (MEIA),
capable of translating high-level tasks expressed in natural language into a
sequence of executable actions. Specifically, we propose a novel Multimodal
Environment Memory (MEM) module, facilitating the integration of embodied
control with large models through the visual-language memory of scenes. This
capability enables MEIA to generate executable action plans based on diverse
requirements and the robot's capabilities. We conduct experiments in a dynamic
virtual cafe environment, utilizing multiple large models through zero-shot
learning, and carefully design scenarios for various situations. The
experimental results showcase the promising performance of our MEIA in various
embodied interactive tasks.

通过多模态环境记忆模块，我们提出了多模态交互式智能体（MEIA），能够将自然语言表达的高级任务转化为可执行动作序列，从而实现了大型模型与具有体现性控制的集成，实验结果展示了 MEIA 在各种交互任务中的良好表现。