We present Causal Amortized Active Structure Learning (CAASL), an active
intervention design policy that can select interventions that are adaptive,
real-time and that does not require access to the likelihood. This policy, an
amortized network based on the transformer, is trained with reinforcement
learning on a simulator of the design environment, and a reward function that
measures how close the true causal graph is to a causal graph posterior
inferred from the gathered data. On synthetic data and a single-cell gene
expression simulator, we demonstrate empirically that the data acquired through
our policy results in a better estimate of the underlying causal graph than
alternative strategies. Our design policy successfully achieves amortized
intervention design on the distribution of the training environment while also
generalizing well to distribution shifts in test-time design environments.
Further, our policy also demonstrates excellent zero-shot generalization to
design environments with dimensionality higher than that during training, and
to intervention types that it has not been trained on.

Causal Amortized Active Structure Learning (CAASL) is a policy that selects adaptive interventions using a transformer-based network trained with reinforcement learning, resulting in a better estimate of the causal graph.

基于深度强化学习的分摊主动因果归纳

Amortized Active Causal Induction with Deep Reinforcement Learning

Existing strategies for managing risks from advanced AI systems often focus
on affecting what AI systems are developed and how they diffuse. However, this
approach becomes less feasible as the number of developers of advanced AI
grows, and impedes beneficial use-cases as well as harmful ones. In response,
we urge a complementary approach: increasing societal adaptation to advanced
AI, that is, reducing the expected negative impacts from a given level of
diffusion of a given AI capability. We introduce a conceptual framework which
helps identify adaptive interventions that avoid, defend against and remedy
potentially harmful uses of AI systems, illustrated with examples in election
manipulation, cyberterrorism, and loss of control to AI decision-makers. We
discuss a three-step cycle that society can implement to adapt to AI.
Increasing society's ability to implement this cycle builds its resilience to
advanced AI. We conclude with concrete recommendations for governments,
industry, and third-parties.

存在的管理高级人工智能系统风险的策略通常关注于影响开发什么样的人工智能系统以及它们如何扩散。然而，随着高级人工智能开发者数量的增加，这种方法变得越来越不可行，并且阻碍了有益的和有害的用例。我们呼吁采用一种补充方法：提高社会对高级人工智能的适应能力，即减少特定级别的人工智能能力扩散所带来的预期负面影响。我们引入了一个概念框架，用于识别避免、防范和纠正潜在有害的人工智能系统使用的适应性干预措施，并结合选举操纵、网络恐怖主义和人工智能决策者失控的示例进行说明。我们讨论了社会可以实施的适应人工智能的三个步骤循环。增强社会实施这一循环的能力将提高社会对高级人工智能的韧性。最后，我们为政府、行业和第三方提出具体建议。

先进人工智能的社会适应

Societal Adaptation to Advanced AI

Personalized adaptive interventions offer the opportunity to increase patient
benefits, however, there are challenges in their planning and implementation.
Once implemented, it is an important question whether personalized adaptive
interventions are indeed clinically more effective compared to a fixed gold
standard intervention. In this paper, we present an innovative N-of-1 trial
study design testing whether implementing a personalized intervention by an
online reinforcement learning agent is feasible and effective. Throughout, we
use a new study on physical exercise recommendations to reduce pain in
endometriosis for illustration. We describe the design of a contextual bandit
recommendation agent and evaluate the agent in simulation studies. The results
show that adaptive interventions add complexity to the design and
implementation process, but have the potential to improve patients' benefits
even if only few observations are available. In order to quantify the expected
benefit, data from previous interventional studies is required. We expect our
approach to be transferable to other interventions and clinical interventions.

个性化适应性干预的规划和实施存在挑战，本研究介绍了在线强化学习代理实施个性化干预的可行性和有效性的 N-of-1 试验设计，结果显示适应性干预能增加患者效益，但设计和实施过程变得更为复杂，同时需要利用之前干预研究的数据来量化预期效益。

设计和评估一个在线增强学习代理器，用于在个体化试验中进行身体锻炼推荐

Designing and evaluating an online reinforcement learning agent for  physical exercise recommendations in N-of-1 trials

Recovering causal relationships from data is an important problem. Using
observational data, one can typically only recover causal graphs up to a Markov
equivalence class and additional assumptions or interventional data are needed
for complete recovery. In this work, under some standard assumptions, we study
causal graph discovery via adaptive interventions with node-dependent
interventional costs. For this setting, we show that no algorithm can achieve
an approximation guarantee that is asymptotically better than linear in the
number of vertices with respect to the verification number; a well-established
benchmark for adaptive search algorithms. Motivated by this negative result, we
define a new benchmark that captures the worst-case interventional cost for any
search algorithm. Furthermore, with respect to this new benchmark, we provide
adaptive search algorithms that achieve logarithmic approximations under
various settings: atomic, bounded size interventions and generalized cost
objectives.

通过研究自适应干预下的因果图发现，提出了一种新的基准，捕捉任何搜索算法的最坏干预成本，并提出了在各种设置下实现对数逼近的自适应搜索算法。