The success of transformer models trained with a language modeling objective
brings a promising opportunity to the reinforcement learning framework.
Decision Transformer is a step towards this direction, showing how to train
transformers with a similar next-step prediction objective on offline data.
Another important development in this area is the recent emergence of
large-scale datasets collected from the internet, such as the ones composed of
tutorial videos with captions where people talk about what they are doing. To
take advantage of this language component, we propose a novel method for
unifying language reasoning with actions in a single policy. Specifically, we
augment a transformer policy with word outputs, so it can generate textual
captions interleaved with actions. When tested on the most challenging task in
BabyAI, with captions describing next subgoals, our reasoning policy
consistently outperforms the caption-free baseline.

提出了一种将 Transformer 模型与强化学习相结合，并在 BabyAI 任务上表现出色的新方法，其中模型输出既包含动作，也包含文本描述。

行动前先思考：将语言推理与行动交错的统一策略

Think Before You Act: Unified Policy for Interleaving Language Reasoning  with Actions

We converted the recently developed BabyAI grid world platform to a
sender/receiver setup in order to test the hypothesis that established deep
reinforcement learning techniques are sufficient to incentivize the emergence
of a grounded discrete communication protocol between generalized agents. This
is in contrast to previous experiments that employed straight-through
estimation or specialized inductive biases. Our results show that these can
indeed be avoided, by instead providing proper environmental incentives.
Moreover, they show that a longer interval between communications incentivized
more abstract semantics. In some cases, the communicating agents adapted to new
environments more quickly than a monolithic agent, showcasing the potential of
emergent communication for transfer learning and generalization in general.

研究将最近开发的 BabyAI 网格世界平台转变为发送者 / 接收者设置，以测试深层强化学习技术是否足以激励建立通用代理之间的基于地面的离散通信协议的假设。结果表明，适当的环境激励确实可以避免测量或特殊归纳偏差的情况，而且更长的通信间隔激励了更抽象的语义。一些情况下，通讯代理比一个单一的代理更快地适应了新环境，展示了新兴通信对于迁移学习和泛化的潜力。