Training agents in cooperative settings offers the promise of AI agents able
to interact effectively with humans (and other agents) in the real world.
Multi-agent reinforcement learning (MARL) has the potential to achieve this
goal, demonstrating success in a series of challenging problems. However,
whilst these advances are significant, the vast majority of focus has been on
the self-play paradigm. This often results in a coordination problem, caused by
agents learning to make use of arbitrary conventions when playing with
themselves. This means that even the strongest self-play agents may have very
low cross-play with other agents, including other initializations of the same
algorithm. In this paper we propose to solve this problem by adapting agent
strategies on the fly, using a posterior belief over the other agents'
strategy. Concretely, we consider the problem of selecting a strategy from a
finite set of previously trained agents, to play with an unknown partner. We
propose an extension of the classic statistical technique, Gibbs sampling, to
update beliefs about other agents and obtain close to optimal ad-hoc
performance. Despite its simplicity, our method is able to achieve strong
cross-play with unseen partners in the challenging card game of Hanabi,
achieving successful ad-hoc coordination without knowledge of the partner's
strategy a priori.

本文提出了使用贝叶斯推断和 Gibbs 采样等技术来解决多智能体强化学习协作中的协调难题，以实现智能体对人类和其他智能体的有效交互，并在 Hanabi 这个复杂的纸牌游戏中表现出了强大的能力。