Self-play is a technique for machine learning in multi-agent systems where a
learning algorithm learns by interacting with copies of itself. Self-play is
useful for generating large quantities of data for learning, but has the
drawback that the agents the learner will face post-training may have
dramatically different behavior than the learner came to expect by interacting
with itself. For the special case of two-player constant-sum games, self-play
that reaches Nash equilibrium is guaranteed to produce strategies that perform
well against any post-training opponent; however, no such guarantee exists for
multi-player games. We show that in games that approximately decompose into a
set of two-player constant-sum games (called polymatrix games) where global
$\epsilon$-Nash equilibria are boundedly far from Nash-equilibria in each
subgame, any no-external-regret algorithm that learns by self-play will produce
a strategy with bounded vulnerability. For the first time, our results identify
a structural property of multi-player games that enable performance guarantees
for the strategies produced by a broad class of self-play algorithms. We
demonstrate our findings through experiments on Leduc poker.

自我对弈是多智能体系统中一种机器学习技术，它通过与自身的副本进行交互学习。本研究发现了多智能体游戏的结构特性，从而为自我对弈算法生成的策略提供了性能保证。

多人游戏中通过多重矩阵可分解性保证自我对弈

Guarantees for Self-Play in Multiplayer Games via Polymatrix  Decomposability

Regularized learning is a fundamental technique in online optimization,
machine learning and many other fields of computer science. A natural question
that arises in these settings is how regularized learning algorithms behave
when faced against each other. We study a natural formulation of this problem
by coupling regularized learning dynamics in zero-sum games. We show that the
system's behavior is Poincar\'e recurrent, implying that almost every
trajectory revisits any (arbitrarily small) neighborhood of its starting point
infinitely often. This cycling behavior is robust to the agents' choice of
regularization mechanism (each agent could be using a different regularizer),
to positive-affine transformations of the agents' utilities, and it also
persists in the case of networked competition, i.e., for zero-sum polymatrix
games.

本文研究了零和博弈中正则化学习动态的耐久性并证明该系统的行为是 Poincaré 循环的