Multi-agent reinforcement learning (MARL) algorithms often suffer from an
exponential sample complexity dependence on the number of agents, a phenomenon
known as \emph{the curse of multiagents}. In this paper, we address this
challenge by investigating sample-efficient model-free algorithms in
\emph{decentralized} MARL, and aim to improve existing algorithms along this
line. For learning (coarse) correlated equilibria in general-sum Markov games,
we propose \emph{stage-based} V-learning algorithms that significantly simplify
the algorithmic design and analysis of recent works, and circumvent a rather
complicated no-\emph{weighted}-regret bandit subroutine. For learning Nash
equilibria in Markov potential games, we propose an independent policy gradient
algorithm with a decentralized momentum-based variance reduction technique. All
our algorithms are decentralized in that each agent can make decisions based on
only its local information. Neither communication nor centralized coordination
is required during learning, leading to a natural generalization to a large
number of agents. We also provide numerical simulations to corroborate our
theoretical findings.

本文针对多智能体强化学习算法在代理数目增多时出现的采样复杂度指数级增长的现象，提出了一些去中心化的学习算法，并在几个关键的方面上做了优化，同时通过数值仿真验证我们理论的有效性。