Deep reinforcement learning (DRL) has shown incredible performance in learning various tasks to the human level. However, unlike human perception, current DRL models connect the entire low-level sensory input to the state-action values rather than exploiting the relationship between and among entities that constitute the sensory input. Because of this difference, DRL needs vast amount of experience samples to learn. In this paper, we propose a Multi-focus Attention Network (MANet) which mimics human ability to spatially abstract the low-level sensory input into multiple entities and attend to them simultaneously. The proposed method first divides the low-level input into several segments which we refer to as partial states. After this segmentation, parallel attention layers attend to the partial states relevant to solving the task. Our model estimates state-action values using these attended partial states. In our experiments, MANet attains highest scores with significantly less experience samples. Additionally, the model shows higher performance compared to the Deep Q-network and the single attention model as benchmarks. Furthermore, we extend our model to attentive communication model for performing multi-agent cooperative tasks. In multi-agent cooperative task experiments, our model shows 20% faster learning than existing state-of-the-art model.

本文提出了一种多焦点注意力网络 (Multi-focus Attention Network, MANet) 模型，通过模仿人类对低级感官输入进行空间抽象并同时关注它们的能力，将其分割成若干个部分状态后，利用并行的注意力层关注与任务有关的部分状态进行状态值估计；实验证明，MANet 在体验采样数显著减少的情况下取得了最高分，并相比 Deep Q-network 和单一注意力模型表现更好，同时在多智能体合作任务中，我们的模型比现有最先进模型的学习速度提高了 20%。

高效深度强化学习的多焦点注意力网络