Communication lays the foundation for cooperation in human society and in
multi-agent reinforcement learning (MARL). Humans also desire to maintain their
privacy when communicating with others, yet such privacy concern has not been
considered in existing works in MARL. To this end, we propose the
\textit{differentially private multi-agent communication} (DPMAC) algorithm,
which protects the sensitive information of individual agents by equipping each
agent with a local message sender with rigorous $(\epsilon,
\delta)$-differential privacy (DP) guarantee. In contrast to directly
perturbing the messages with predefined DP noise as commonly done in
privacy-preserving scenarios, we adopt a stochastic message sender for each
agent respectively and incorporate the DP requirement into the sender, which
automatically adjusts the learned message distribution to alleviate the
instability caused by DP noise. Further, we prove the existence of a Nash
equilibrium in cooperative MARL with privacy-preserving communication, which
suggests that this problem is game-theoretically learnable. Extensive
experiments demonstrate a clear advantage of DPMAC over baseline methods in
privacy-preserving scenarios.

提出了一种差分隐私的多智能体通信 (DPMAC) 算法用于多智能体强化学习中，该算法通过为每个智能体配备具有严格 (ε, δ)- 差分隐私保证的本地消息发送器来保护个体智能体的敏感信息，并在合作性多智能体强化学习中证明了纳什均衡 (Nash equilibrium) 的存在，从而表明该问题是可以在博弈理论层面上学习的。通过大量实验证明了 DPMAC 在隐私保护场景中相对于基准方法的明显优势。