Curriculum reinforcement learning (CRL) aims to speed up learning of a task
by changing gradually the difficulty of the task from easy to hard through
control of factors such as initial state or environment dynamics. While
automating CRL is well studied in the single-agent setting, in multi-agent
reinforcement learning (MARL) an open question is whether control of the number
of agents with other factors in a principled manner is beneficial, prior
approaches typically relying on hand-crafted heuristics. In addition, how the
tasks evolve as the number of agents changes remains understudied, which is
critical for scaling to more challenging tasks. We introduce self-paced MARL
(SPMARL) that enables optimizing the number of agents with other environment
factors in a principled way, and, show that usual assumptions such as that
fewer agents make the task always easier are not generally valid. The
curriculum induced by SPMARL reveals the evolution of tasks w.r.t. number of
agents and experiments show that SPMARL improves the performance when the
number of agents sufficiently influences task difficulty.

本文介绍了自适应多智能体强化学习 (Self-paced MARL)，它可以根据任务的难度自动优化智能体数量，实验结果显示该方法在智能体数量对任务难度有足够影响时可以提高任务表现。