Many recent language model (LM) interpretability studies have adopted the
circuits framework, which aims to find the minimal computational subgraph, or
circuit, that explains LM behavior on a given task. Most studies determine
which edges belong in a LM's circuit by performing causal interventions on each
edge independently, but this scales poorly with model size. Edge attribution
patching (EAP), gradient-based approximation to interventions, has emerged as a
scalable but imperfect solution to this problem. In this paper, we introduce a
new method - EAP with integrated gradients (EAP-IG) - that aims to better
maintain a core property of circuits: faithfulness. A circuit is faithful if
all model edges outside the circuit can be ablated without changing the model's
performance on the task; faithfulness is what justifies studying circuits,
rather than the full model. Our experiments demonstrate that circuits found
using EAP are less faithful than those found using EAP-IG, even though both
have high node overlap with circuits found previously using causal
interventions. We conclude more generally that when using circuits to compare
the mechanisms models use to solve tasks, faithfulness, not overlap, is what
should be measured.

通过引入 EAP-IG 方法，本文旨在更好地保持电路的核心属性：忠诚度。我们的实验证明，使用 EAP 找到的电路比使用 EAP-IG 找到的电路更不忠诚，尽管两者在与之前使用因果干预发现的电路的节点重叠方面都很高。总之，在使用电路来比较模型解决任务的机制时，应该衡量忠诚度而不是重叠程度。