A pivotal aim in contemporary AI research is to develop agents proficient in
multi-agent coordination, enabling effective collaboration with both humans and
other systems. Large Language Models (LLMs), with their notable ability to
understand, generate, and interpret language in a human-like manner, stand out
as promising candidates for the development of such agents. In this study, we
build and assess the effectiveness of agents crafted using LLMs in various
coordination scenarios. We introduce the LLM-Coordination (LLM-Co) Framework,
specifically designed to enable LLMs to play coordination games. With the
LLM-Co framework, we conduct our evaluation with three game environments and
organize the evaluation into five aspects: Theory of Mind, Situated Reasoning,
Sustained Coordination, Robustness to Partners, and Explicit Assistance. First,
the evaluation of the Theory of Mind and Situated Reasoning reveals the
capabilities of LLM to infer the partner's intention and reason actions
accordingly. Then, the evaluation around Sustained Coordination and Robustness
to Partners further showcases the ability of LLMs to coordinate with an unknown
partner in complex long-horizon tasks, outperforming Reinforcement Learning
baselines. Lastly, to test Explicit Assistance, which refers to the ability of
an agent to offer help proactively, we introduce two novel layouts into the
Overcooked-AI benchmark, examining if agents can prioritize helping their
partners, sacrificing time that could have been spent on their tasks. This
research underscores the promising capabilities of LLMs in sophisticated
coordination environments and reveals the potential of LLMs in building strong
real-world agents for multi-agent coordination.

该研究介绍了使用大型语言模型在多智能体协调方面的有效性评估，构建了 LLM-Co 框架来使 LLMs 能够进行协调游戏，并展示了 LLMs 在不同方面的评估结果，从而强调了 LLMs 在复杂协调环境中的潜力。

评估大型语言模型中的多智能体协调能力

Evaluating Multi-Agent Coordination Abilities in Large Language Models

We present volume analyses of Multiplicative Weights Updates (MWU) and
Optimistic Multiplicative Weights Updates (OMWU) in zero-sum as well as
coordination games. Such analyses provide new insights into these game
dynamical systems, which seem hard to achieve via the classical techniques
within Computer Science and Machine Learning.
The first step is to examine these dynamics not in their original space
(simplex of actions) but in a dual space (aggregate payoff space of actions).
The second step is to explore how the volume of a set of initial conditions
evolves over time when it is pushed forward according to the algorithm. This is
reminiscent of approaches in Evolutionary Game Theory where replicator
dynamics, the continuous-time analogue of MWU, is known to always preserve
volume in all games. Interestingly, when we examine discrete-time dynamics,
both the choice of the game and the choice of the algorithm play a critical
role. So whereas MWU expands volume in zero-sum games and is thus Lyapunov
chaotic, we show that OMWU contracts volume, providing an alternative
understanding for its known convergent behavior. However, we also prove a
no-free-lunch type of theorem, in the sense that when examining coordination
games the roles are reversed: OMWU expands volume exponentially fast, whereas
MWU contracts.
Using these tools, we prove two novel, rather negative properties of MWU in
zero-sum games: (1) Extremism: even in games with unique fully mixed Nash
equilibrium, the system recurrently gets stuck near pure-strategy profiles,
despite them being clearly unstable from game theoretic perspective. (2)
Unavoidability: given any set of good points (with your own interpretation of
"good"), the system cannot avoid bad points indefinitely.

本文研究了 MWU 和 OMWU 在零和博弈和协调博弈中的体积分析，并通过对初值集合的变换证明了其新颖的收敛或发散特性。

游戏中的混沌、极端主义和乐观主义：学习的量化分析

Chaos, Extremism and Optimism: Volume Analysis of Learning in Games

We consider the problem of distributed convergence to efficient outcomes in
coordination games through dynamics based on aspiration learning. Under
aspiration learning, a player continues to play an action as long as the
rewards received exceed a specified aspiration level. Here, the aspiration
level is a fading memory average of past rewards, and these levels also are
subject to occasional random perturbations. A player becomes dissatisfied
whenever a received reward is less than the aspiration level, in which case the
player experiments with a probability proportional to the degree of
dissatisfaction. Our first contribution is the characterization of the
asymptotic behavior of the induced Markov chain of the iterated process in
terms of an equivalent finite-state Markov chain. We then characterize
explicitly the behavior of the proposed aspiration learning in a generalized
version of coordination games, examples of which include network formation and
common-pool games. In particular, we show that in generic coordination games
the frequency at which an efficient action profile is played can be made
arbitrarily large. Although convergence to efficient outcomes is desirable, in
several coordination games, such as common-pool games, attainability of fair
outcomes, i.e., sequences of plays at which players experience highly rewarding
returns with the same frequency, might also be of special interest. To this
end, we demonstrate through analysis and simulations that aspiration learning
also establishes fair outcomes in all symmetric coordination games, including
common-pool games.

以心怀期望的加速学习方法解决协调和公共池游戏的分布式收敛问题，并在对称协调游戏中建立公平结果。