The challenge of developing powerful and general Reinforcement Learning (RL) agents has received increasing attention in recent years. Much of this effort has focused on the single-agent setting, in which an agent maximizes a predefined extrinsic reward function. However, a long-term question inevitably arises: how will such independent agents cooperate when they are continually learning and acting in a shared multi-agent environment? Observing that humans often provide incentives to influence others' behavior, we propose to equip each RL agent in a multi-agent environment with the ability to give rewards directly to other agents, using a learned incentive function. Each agent learns its own incentive function by explicitly accounting for its impact on the learning of recipients and, through them, the impact on its own extrinsic objective. We demonstrate in experiments that such agents significantly outperform standard RL and opponent-shaping agents in challenging general-sum Markov games, often by finding a near-optimal division of labor. Our work points toward more opportunities and challenges along the path to ensure the common good in a multi-agent future.

本文提出了在多个智能体环境中，为每个RL 智能体提供直接向其它智能体给予奖励的能力，并通过学习后的激励函数影响其它智能体，从而达到协作的目的。实验结果显示，在 challenging general-sum Markov games 中，相对于标准RL和对手建模代理，这种方法在寻找最优的分工方面取得了巨大的成功。

学习如何激励其他学习智能体