Identification and analysis of symmetrical patterns in the natural world have led to significant discoveries across various scientific fields, such as the formulation of gravitational laws in physics and advancements in the study of chemical structures. In this paper, we focus on exploiting Euclidean symmetries inherent in certain cooperative multi-agent reinforcement learning (MARL) problems and prevalent in many applications. We begin by formally characterizing a subclass of Markov games with a general notion of symmetries that admits the existence of symmetric optimal values and policies. Motivated by these properties, we design neural network architectures with symmetric constraints embedded as an inductive bias for multi-agent actor-critic methods. This inductive bias results in superior performance in various cooperative MARL benchmarks and impressive generalization capabilities such as zero-shot learning and transfer learning in unseen scenarios with repeated symmetric patterns. The code is available at: https://github.com/dchen48/E3AC.

本文重点研究了自然界中对称模式的识别和分析，在物理学中导致了引力定律的制定和化学结构研究的进展。我们着眼于利用某些协同多智能体强化学习问题中固有的欧几里得对称性，该问题在许多应用中普遍存在。我们首先形式化地表征了一类具有对称最优值和策略存在性的马尔科夫博弈的子类。在这些属性的基础上，我们设计了具有对称约束的神经网络架构，作为多智能体演员-评论家方法的归纳偏见。这种归纳偏见在各种协同多智能体强化学习基准测试中表现出优越的性能，以及在具有重复对称模式的未见场景中进行的零样本学习和迁移学习等令人印象深刻的泛化能力。代码可在此 https URL 获取。

${ m E}(3)$-Equivariant Actor-Critic 合作多智能体强化学习方法