Whereas machine learning models typically learn language by directly training
on language tasks (e.g., next-word prediction), language emerges in human
children as a byproduct of solving non-language tasks (e.g., acquiring food).
Motivated by this observation, we ask: can embodied reinforcement learning (RL)
agents also indirectly learn language from non-language tasks? Learning to
associate language with its meaning requires a dynamic environment with varied
language. Therefore, we investigate this question in a multi-task environment
with language that varies across the different tasks. Specifically, we design
an office navigation environment, where the agent's goal is to find a
particular office, and office locations differ in different buildings (i.e.,
tasks). Each building includes a floor plan with a simple language description
of the goal office's location, which can be visually read as an RGB image when
visited. We find RL agents indeed are able to indirectly learn language. Agents
trained with current meta-RL algorithms successfully generalize to reading
floor plans with held-out layouts and language phrases, and quickly navigate to
the correct office, despite receiving no direct language supervision.

该研究探讨了机器学习模型是否能够通过非语言任务间接学习语言，设计了一个含有变化语言环境和目的地的综合任务环境，通过元强化学习算法，培养出能够正确理解渐新语言和 floor plans 的智能体。

元强化学习的附带简易具身语言学习

Simple Embodied Language Learning as a Byproduct of Meta-Reinforcement  Learning

We investigate the use of natural language to drive the generalization of
control policies and introduce the new multi-task environment Messenger with
free-form text manuals describing the environment dynamics. Unlike previous
work, Messenger does not assume prior knowledge connecting text and state
observations $-$ the control policy must simultaneously ground the game manual
to entity symbols and dynamics in the environment. We develop a new model, EMMA
(Entity Mapper with Multi-modal Attention) which uses an entity-conditioned
attention module that allows for selective focus over relevant descriptions in
the manual for each entity in the environment. EMMA is end-to-end
differentiable and learns a latent grounding of entities and dynamics from text
to observations using only environment rewards. EMMA achieves successful
zero-shot generalization to unseen games with new dynamics, obtaining a 40%
higher win rate compared to multiple baselines. However, win rate on the
hardest stage of Messenger remains low (10%), demonstrating the need for
additional work in this direction.

本文研究了利用自然语言来推动控制策略的泛化，并引入了一个新的多任务环境 Messenger，其中包括自由形式的文本手册，描述环境动态。使用实体 - 条件的注意力模块，作者提出了一个新的模型 EMMA，它可以从文本到观察值学习实体和动态之间的潜在关系，并实现了成功的零样本泛化。