This report outlines work by the Modeling Transformative AI Risk (MTAIR)
project, an attempt to map out the key hypotheses, uncertainties, and
disagreements in debates about catastrophic risks from advanced AI, and the
relationships between them. This builds on an earlier diagram by Ben Cottier
and Rohin Shah which laid out some of the crucial disagreements ("cruxes")
visually, with some explanation. Based on an extensive literature review and
engagement with experts, the report explains a model of the issues involved,
and the initial software-based implementation that can incorporate probability
estimates or other quantitative factors to enable exploration, planning, and/or
decision support. By gathering information from various debates and discussions
into a single more coherent presentation, we hope to enable better discussions
and debates about the issues involved.
The model starts with a discussion of reasoning via analogies and general
prior beliefs about artificial intelligence. Following this, it lays out a
model of different paths and enabling technologies for high-level machine
intelligence, and a model of how advances in the capabilities of these systems
might proceed, including debates about self-improvement, discontinuous
improvements, and the possibility of distributed, non-agentic high-level
intelligence or slower improvements. The model also looks specifically at the
question of learned optimization, and whether machine learning systems will
create mesa-optimizers. The impact of different safety research on the previous
sets of questions is then examined, to understand whether and how research
could be useful in enabling safer systems. Finally, we discuss a model of
different failure modes and loss of control or takeover scenarios.

该报告旨在通过梳理先前的分歧并建立一个涵盖概率估计和定量因素的模型，解释了 AI 灾难性风险相关问题及其关系；并讨论了不同类型的人工智能、技术和其先进程度，以及安全研究和失控 / 接管场景等因素的影响。