Obtaining human-interpretable explanations of large, general-purpose language
models is an urgent goal for AI safety. However, it is just as important that
our interpretability methods are faithful to the causal dynamics underlying
model behavior and able to robustly generalize to unseen inputs. Distributed
Alignment Search (DAS) is a powerful gradient descent method grounded in a
theory of causal abstraction that uncovered perfect alignments between
interpretable symbolic algorithms and small deep learning models fine-tuned for
specific tasks. In the present paper, we scale DAS significantly by replacing
the remaining brute-force search steps with learned parameters -- an approach
we call DAS. This enables us to efficiently search for interpretable causal
structure in large language models while they follow instructions. We apply DAS
to the Alpaca model (7B parameters), which, off the shelf, solves a simple
numerical reasoning problem. With DAS, we discover that Alpaca does this by
implementing a causal model with two interpretable boolean variables.
Furthermore, we find that the alignment of neural representations with these
variables is robust to changes in inputs and instructions. These findings mark
a first step toward deeply understanding the inner-workings of our largest and
most widely deployed language models.

本文介绍了基于因果抽象的分布式对齐搜索方法（Distributed Alignment Search，DAS），通过替换搜索过程并训练参数，能够高效地在大型语言模型中搜索可解释的因果结构。在应用 DAS 于 Alpaca 模型时，我们发现它通过实现一个具有两个可解释的布尔变量的因果模型来解决一个简单的数值推理问题，并且神经表示与这些变量的对齐具有鲁棒性。这些发现标志着我们深入了解最大、最广泛使用的语言模型内部运行机制的第一步。

规模上的可解释性：在 Alpaca 中识别因果机制

Interpretability at Scale: Identifying Causal Mechanisms in Alpaca

Causal abstraction is a promising theoretical framework for explainable
artificial intelligence that defines when an interpretable high-level causal
model is a faithful simplification of a low-level deep learning system.
However, existing causal abstraction methods have two major limitations: they
require a brute-force search over alignments between the high-level model and
the low-level one, and they presuppose that variables in the high-level model
will align with disjoint sets of neurons in the low-level one. In this paper,
we present distributed alignment search (DAS), which overcomes these
limitations. In DAS, we find the alignment between high-level and low-level
models using gradient descent rather than conducting a brute-force search, and
we allow individual neurons to play multiple distinct roles by analyzing
representations in non-standard bases-distributed representations. Our
experiments show that DAS can discover internal structure that prior approaches
miss. Overall, DAS removes previous obstacles to conducting causal abstraction
analyses and allows us to find conceptual structure in trained neural nets.

本文介绍了一种名为分布式对齐搜索（DAS）的方法，它使用梯度下降来找出高层和低层模型之间的对齐，并允许单个神经元在非标准基中扮演多个不同的角色，从而发现了其他方法所错过的内在结构，为进行因果抽象分析消除了前期的限制。