Artificially intelligent agents deployed in the real-world will require the
ability to reliably \textit{cooperate} with humans (as well as other,
heterogeneous AI agents). To provide formal guarantees of successful
cooperation, we must make some assumptions about how partner agents could
plausibly behave. Any realistic set of assumptions must account for the fact
that other agents may be just as adaptable as our agent is. In this work, we
consider the problem of cooperating with a \textit{population} of agents in a
finitely-repeated, two player general-sum matrix game with private utilities.
Two natural assumptions in such settings are that: 1) all agents in the
population are individually rational learners, and 2) when any two members of
the population are paired together, with high-probability they will achieve at
least the same utility as they would under some Pareto efficient equilibrium
strategy. Our results first show that these assumptions alone are insufficient
to ensure \textit{zero-shot} cooperation with members of the target population.
We therefore consider the problem of \textit{learning} a strategy for
cooperating with such a population using prior observations its members
interacting with one another. We provide upper and lower bounds on the number
of samples needed to learn an effective cooperation strategy. Most importantly,
we show that these bounds can be much stronger than those arising from a
"naive'' reduction of the problem to one of imitation learning.

人工智能代理在现实世界中的部署需要与人类（以及其他异构的 AI 代理）可靠地合作。为了提供成功合作的形式化保证，我们必须对合作伙伴代理的行为做一些合理的假设。这项工作研究了在一个有限重复的、两个玩家的一般化总和矩阵游戏中与一个代理人群合作的问题，通过个体理性学习者的假设和在某个 Pareto 有效均衡策略下高概率实现至少与该策略相同效用，我们证明了这些假设本身不足以保证与目标人群成员的零 - shot 合作。因此，我们考虑了通过先前观察到的人群成员相互作用来学习与这样的人群合作的策略问题，并给出了学习有效合作策略所需样本数量的上限和下限。最重要的是，我们证明了这些界限可以比通过一种 “天真” 的问题简化到模仿学习中产生的界限更强。

学习与社会理性个体群体合作的复杂度

On the Complexity of Learning to Cooperate with Populations of Socially  Rational Agents

The problem is considered of optimizing discrete parameters in the presence
of constraints. We use the stochastic sigmoid with temperature and put forward
the new adaptive gradient method CONGA. The search for an optimal solution is
carried out by a population of individuals. Each of them varies according to
gradients of the 'environment' and is characterized by two temperature
parameters with different annealing schedules. Unadapted individuals die, and
optimal ones interbreed, the result is directed evolutionary dynamics. The
proposed method is illustrated using the well-known combinatorial problem for
optimal packing of a backpack (0-1 KP).

优化离散参数存在约束问题，我们使用带有温度的随机 S 型函数，并提出了新的自适应梯度方法 CONGA。通过一个个体群体来搜索最优解，每个个体根据 ' 环境 ' 的梯度变化，并由具有不同退火进程的两个温度参数来描述。未经适应的个体消亡，而最优个体繁殖，其结果是定向的进化动力学。我们利用著名的背包完全装配（0-1 KP）问题来说明所提出的方法。

使用自适应梯度方法和定向进化优化离散参数

Optimization of Discrete Parameters Using the Adaptive Gradient Method  and Directed Evolution

Non-cooperative and cooperative games with a very large number of players
have many applications but remain generally intractable when the number of
players increases. Introduced by Lasry and Lions, and Huang, Caines and
Malhamé, Mean Field Games (MFGs) rely on a mean-field approximation to allow
the number of players to grow to infinity. Traditional methods for solving
these games generally rely on solving partial or stochastic differential
equations with a full knowledge of the model. Recently, Reinforcement Learning
(RL) has appeared promising to solve complex problems. By combining MFGs and
RL, we hope to solve games at a very large scale both in terms of population
size and environment complexity. In this survey, we review the quickly growing
recent literature on RL methods to learn Nash equilibria in MFGs. We first
identify the most common settings (static, stationary, and evolutive). We then
present a general framework for classical iterative methods (based on
best-response computation or policy evaluation) to solve MFGs in an exact way.
Building on these algorithms and the connection with Markov Decision Processes,
we explain how RL can be used to learn MFG solutions in a model-free way. Last,
we present numerical illustrations on a benchmark problem, and conclude with
some perspectives.

该论文综述如何运用强化学习和均值场博弈来解决无法通过传统方法计算的大规模人口问题，并针对静态、平稳和演变三个最常见的情境，提出一套基于最佳策略和策略评估的迭代方法以及没有模型计算的强化学习解决方案。

学习平均场博弈：一项调查

Learning Mean Field Games: A Survey

$\textit{No man is an island.}$ Humans communicate with a large community by
coordinating with different interlocutors within short conversations. This
ability has been understudied by the research on building neural communicative
agents. We study the task of few-shot $\textit{language coordination}$: agents
quickly adapting to their conversational partners' language abilities.
Different from current communicative agents trained with self-play, we require
the lead agent to coordinate with a $\textit{population}$ of agents with
different linguistic abilities, quickly adapting to communicate with unseen
agents in the population. This requires the ability to model the partner's
beliefs, a vital component of human communication. Drawing inspiration from
theory-of-mind (ToM; Premack& Woodruff (1978)), we study the effect of the
speaker explicitly modeling the listeners' mental states. The speakers, as
shown in our experiments, acquire the ability to predict the reactions of their
partner, which helps it generate instructions that concisely express its
communicative goal. We examine our hypothesis that the instructions generated
with ToM modeling yield better communication performance in both a referential
game and a language navigation task. Positive results from our experiments hint
at the importance of explicitly modeling communication as a socio-pragmatic
progress.

该研究探讨了几颗子任务的语言协调，通过群体与不同的语言能力代理进行快速交流，提出了使用 ToM 建立代理人之间交流的重要性，实验表明使用 ToM 建立指令可以更好地执行交流任务。

通过建模心理理论实现少样本语言协调

Few-shot Language Coordination by Modeling Theory of Mind

We report on the population of the 47 compact binary mergers detected with a
false-alarm rate 1/yr in the second LIGO--Virgo Gravitational-Wave Transient
Catalog, GWTC-2. We observe several characteristics of the merging binary black
hole (BBH) population not discernible until now. First, we find that the
primary mass spectrum contains structure beyond a power-law with a sharp
high-mass cut-off; it is more consistent with a broken power law with a break
at $39.7^{+20.3}_{-9.1}\,M_\odot$, or a power law with a Gaussian feature
peaking at $33.1^{+4.0}_{-5.6}\,M_\odot$ (90\% credible interval). While the
primary mass distribution must extend to $\sim65\,M_\odot$ or beyond, only
$2.9^{+3.5}_{1.7}\%$ of systems have primary masses greater than $45\,M_\odot$.
Second, we find that a fraction of BBH systems have component spins misaligned
with the orbital angular momentum, giving rise to precession of the orbital
plane. Moreover, 12% to 44% of BBH systems have spins tilted by more than
$90^\circ$, giving rise to a negative effective inspiral spin parameter
$\chi_\mathrm{eff}$. Under the assumption that such systems can only be formed
by dynamical interactions, we infer that between 25% and 93% of BBH with
non-vanishing $|\chi_\mathrm{eff}| > 0.01$ are dynamically assembled. Third, we
estimate merger rates, finding $\mathcal{R}_\text{BBH} = 23.9^{+14.3}_{8.6}$
Gpc$^{-3}$ yr$^{-1}$ for BBH and $\mathcal{R}_\text{BNS}= 320^{+490}_{-240}$
Gpc$^{-3}$ yr$^{-1}$ for binary neutron stars. We find that the BBH rate likely
increases with redshift ($85\%$ credibility), but not faster than the
star-formation rate ($86\%$ credibility). Additionally, we examine recent
exceptional events in the context of our population models, finding that the
asymmetric masses of GW190412 and the high component masses of GW190521 are
consistent with our models, but the low secondary mass of GW190814 makes it an
outlier.

论文研究了二进制黑洞的人口统计数据，在此基础上发现了多种黑洞种群的特征，包括不符合幂律分布的质量谱结构以及变异的自转轨道等特点，并推导了黑洞及中子星二元体合并的概率。

第二个 LIGO-Virgo 引力波瞬变目录中紧凑天体的群体特性

Population Properties of Compact Objects from the Second LIGO-Virgo  Gravitational-Wave Transient Catalog

Shortly after a new class of objects is discovered, the attention shifts from
the properties of the individual sources to the question of their origin: do
all sources come from the same underlying population, or several populations
are required? What are the properties of these populations? As the detection of
gravitational waves is becoming routine and the size of the event catalog
increases, finer and finer details of the astrophysical distribution of compact
binaries are now within our grasp. This Chapter presents a pedagogical
introduction to the main statistical tool required for these analyses:
hierarchical Bayesian inference in the presence of selection effects. All key
equations are obtained from first principles, followed by two examples of
increasing complexity. Although many remarks made in this Chapter refer to
gravitational-wave astronomy, the write-up is generic enough to be useful to
researchers and graduate students from other fields.

本章介绍了一种用于分析紧密联星天体分布的统计学工具：在选择效应存在的情况下进行分层贝叶斯推断。该方法适用于各学科的研究者和研究生，本章从第一原理出发给出了该方法的所有关键方程，并给出了两个复杂度不断增加的应用实例。

在选择效应存在下推断紧致双星族群的属性

Inferring the properties of a population of compact binaries in presence  of selection effects

In this paper, we investigate a new approach to Population, Intervention and
Outcome (PIO) element detection, a common task in Evidence Based Medicine
(EBM). The purpose of this study is two-fold: to build a training dataset for
PIO element detection with minimum redundancy and ambiguity and to investigate
possible options in utilizing state of the art embedding methods for the task
of PIO element detection. For the former purpose, we build a new and improved
dataset by investigating the shortcomings of previously released datasets. For
the latter purpose, we leverage the state of the art text embedding,
Bidirectional Encoder Representations from Transformers (BERT), and build a
multi-label classifier. We show that choosing a domain specific pre-trained
embedding further optimizes the performance of the classifier. Furthermore, we
show that the model could be enhanced by using ensemble methods and boosting
techniques provided that features are adequately chosen.

本研究探究了一种新的人口、干预和结果（PIO）元素检测方法，特别关注证据医学（EBM）中的这一任务。本研究目的是建立一个训练数据集，以便最大限度地减少冗余和歧义，并探讨利用最先进的嵌入方法进行 PIO 元素检测的可能选项。我们利用最先进的文本嵌入 BERT 建立了一个多标签分类器，这进一步优化了分类器的性能，同时表明模型可以通过使用集成方法和提升技术来提高。

利用上下文嵌入增强医学文本中 PIO 元素的检测

Enhancing PIO Element Detection in Medical Text Using Contextualized  Embedding

We consider the spread of infectious disease through contact networks of
Configuration Model type. We assume that the disease spreads through contacts
and infected individuals recover into an immune state. We discuss a number of
existing mathematical models used to investigate this system, and show
relations between the underlying assumptions of the models. In the process we
offer simplifications of some of the existing models. The distinctions between
the underlying assumptions are subtle, and in many if not most cases this
subtlety is irrelevant. Indeed, under appropriate conditions the models are
equivalent. We compare the benefits and disadvantages of the different models,
and discuss their application to other populations (\emph{e.g.,} clustered
networks). Finally we discuss ongoing challenges for network-based epidemic
modeling.

本文研究了基于网络结构模型下的传染病传播，讨论了现有的数学模型的优势和劣势，并探讨了它们在其他人口群体中的应用和现有挑战。

网络中的疫情传播：现有方法和当前挑战

Epidemic spread in networks: Existing methods and current challenges

Entropy and information provide natural measures of correlation among
elements in a network. We construct here the information theoretic analog of
connected correlation functions: irreducible $N$--point correlation is measured
by a decrease in entropy for the joint distribution of $N$ variables relative
to the maximum entropy allowed by all the observed $N-1$ variable
distributions. We calculate the ``connected information'' terms for several
examples, and show that it also enables the decomposition of the information
that is carried by a population of elements about an outside source.

本文研究熵和信息在网络中的相关性，建立了连接性相关函数的信息理论模型，并利用其分解了元素群体对外部源的信息。