In a multi-agent system, agents share their local observations to gain global
situational awareness for decision making and collaboration using a message
passing system. When to send a message, how to encode a message, and how to
leverage the received messages directly affect the effectiveness of the
collaboration among agents. When training a multi-agent cooperative game using
reinforcement learning (RL), the message passing system needs to be optimized
together with the agent policies. This consequently increases the model's
complexity and poses significant challenges to the convergence and performance
of learning. To address this issue, we propose the Belief-map Assisted
Multi-agent System (BAMS), which leverages a neuro-symbolic belief map to
enhance training. The belief map decodes the agent's hidden state to provide a
symbolic representation of the agent's understanding of the environment and
other agent's status. The simplicity of symbolic representation allows the
gathering and comparison of the ground truth information with the belief, which
provides an additional channel of feedback for the learning. Compared to the
sporadic and delayed feedback coming from the reward in RL, the feedback from
the belief map is more consistent and reliable. Agents using BAMS can learn a
more effective message passing network to better understand each other,
resulting in better performance in a cooperative predator and prey game with
varying levels of map complexity and compare it to previous multi-agent message
passing models. The simulation results showed that BAMS reduced training epochs
by 66\%, and agents who apply the BAMS model completed the game with 34.62\%
fewer steps on average.

多智能体系统中，使用消息传递系统共享本地观察以获得全局情境感知，进而影响智能体间协作的有效性。本文提出 Belief-map Assisted Multi-agent System (BAMS)，利用神经符号信念映射来增强训练，通过简单的符号表示实现从反馈到学习的额外通道。与强化学习中来自奖励的间歇性和延迟反馈相比，信念映射提供更一致可靠的反馈，BAMS 模型在合作狩猎游戏中表现更好。实验结果显示，BAMS 减少了 66% 的训练周期，使用 BAMS 的智能体平均少完成了 34.62% 的步骤。