Modern Reinforcement Learning (RL) algorithms are able to outperform humans in a wide variety of tasks. Multi-agent reinforcement learning (MARL) settings present additional challenges, and successful cooperation in mixed-motive groups of agents depends on a delicate balancing act between individual and group objectives. Social conventions and norms, often inspired by human institutions, are used as tools for striking this balance. In this paper, we examine a fundamental, well-studied social convention that underlies cooperation in both animal and human societies: Dominance hierarchies. We adapt the ethological theory of dominance hierarchies to artificial agents, borrowing the established terminology and definitions with as few amendments as possible. We demonstrate that populations of RL agents, operating without explicit programming or intrinsic rewards, can invent, learn, enforce, and transmit a dominance hierarchy to new populations. The dominance hierarchies that emerge have a similar structure to those studied in chickens, mice, fish, and other species.

现代强化学习算法在各种任务中能够超越人类表现。本文研究了多智能体强化学习环境中的一个基本社会约定：优势等级体系。通过人工智能代理，无需明确编程或内在奖励，我们证明了代理群体能够发明、学习、强化和传播优势等级体系，其结构与鸡、老鼠、鱼类和其他物种的研究相似。

强化学习代理的紧急主导层级