Many settings of interest involving humans and machines -- from virtual
personal assistants to autonomous vehicles -- can naturally be modelled as
principals (humans) delegating to agents (machines), which then interact with
each other on their principals' behalf. We refer to these multi-principal,
multi-agent scenarios as delegation games. In such games, there are two
important failure modes: problems of control (where an agent fails to act in
line their principal's preferences) and problems of cooperation (where the
agents fail to work well together). In this paper we formalise and analyse
these problems, further breaking them down into issues of alignment (do the
players have similar preferences?) and capabilities (how competent are the
players at satisfying those preferences?). We show -- theoretically and
empirically -- how these measures determine the principals' welfare, how they
can be estimated using limited observations, and thus how they might be used to
help us design more aligned and cooperative AI systems.

本文中，我们正式分析了代理人协作中的控制问题、合作问题、对齐问题和能力问题，以及这些问题对委托人利益的影响，并展示了如何通过有限观察来估计这些指标，并为设计更协调和合作的 AI 系统提供帮助。

委派博弈中的合作与控制

Cooperation and Control in Delegation Games

It is likely that AI systems driven by pre-trained language models (PLMs)
will increasingly be used to assist humans in high-stakes interactions with
other agents, such as negotiation or conflict resolution. Consistent with the
goals of Cooperative AI \citep{dafoe_open_2020}, we wish to understand and
shape the multi-agent behaviors of PLMs in a pro-social manner. An important
first step is the evaluation of model behaviour across diverse cooperation
problems. Since desired behaviour in an interaction depends upon precise
game-theoretic structure, we focus on generating scenarios with particular
structures with both crowdworkers and a language model. Our work proceeds as
follows. First, we discuss key methodological issues in the generation of
scenarios corresponding to particular game-theoretic structures. Second, we
employ both crowdworkers and a language model to generate such scenarios. We
find that the quality of generations tends to be mediocre in both cases. We
additionally get both crowdworkers and a language model to judge whether given
scenarios align with their intended game-theoretic structure, finding mixed
results depending on the game. Third, we provide a dataset of scenario based on
our data generated. We provide both quantitative and qualitative evaluations of
UnifiedQA and GPT-3 on this dataset. We find that instruct-tuned models tend to
act in a way that could be perceived as cooperative when scaled up, while other
models seemed to have flat scaling trends.

本文探讨了基于预训练语言模型的 AI 系统在协作问题上的行为，并通过众包和模型生成场景进行评估和测试，最后提供了一组生成场景的数据集和模型评估结果。