Human intelligence's adaptability is remarkable, allowing us to adjust to new tasks and multi-modal environments swiftly. This skill is evident from a young age as we acquire new abilities and solve problems by imitating others or following natural language instructions. The research community is actively pursuing the development of interactive "embodied agents" that can engage in natural conversations with humans and assist them with real-world tasks. These agents must possess the ability to promptly request feedback in case communication breaks down or instructions are unclear. Additionally, they must demonstrate proficiency in learning new vocabulary specific to a given domain. In this paper, we made the following contributions: (1) a crowd-sourcing tool for collecting grounded language instructions; (2) the largest dataset of grounded language instructions; and (3) several state-of-the-art baselines. These contributions are suitable as a foundation for further research.

本文介绍了一种交互式的“体现代理”系统，它具有适应性，能够有效地处理自然语言指令，并提供反馈。同时，还介绍了一种用于收集有关该系统的大量文本指令的众包工具，以及该系统具有学习能力的数据集和基线模型。

通过交互式基于语境的语言指导重新定义具有代表性的代理人能力，改变以人为中心的AI协作