The path to interpreting a language model often proceeds via analysis of
circuits -- sparse computational subgraphs of the model that capture specific
aspects of its behavior. Recent work has automated the task of discovering
circuits. Yet, these methods have practical limitations, as they rely either on
inefficient search algorithms or inaccurate approximations. In this paper, we
frame automated circuit discovery as an optimization problem and propose *Edge
Pruning* as an effective and scalable solution. Edge Pruning leverages
gradient-based pruning techniques, but instead of removing neurons or
components, it prunes the \emph{edges} between components. Our method finds
circuits in GPT-2 that use less than half the number of edges compared to
circuits found by previous methods while being equally faithful to the full
model predictions on standard circuit-finding tasks. Edge Pruning is efficient
even with as many as 100K examples, outperforming previous methods in speed and
producing substantially better circuits. It also perfectly recovers the
ground-truth circuits in two models compiled with Tracr. Thanks to its
efficiency, we scale Edge Pruning to CodeLlama-13B, a model over 100x the scale
that prior methods operate on. We use this setting for a case study comparing
the mechanisms behind instruction prompting and in-context learning. We find
two circuits with more than 99.96% sparsity that match the performance of the
full model and reveal that the mechanisms in the two settings overlap
substantially. Our case study shows that Edge Pruning is a practical and
scalable tool for interpretability and sheds light on behaviors that only
emerge in large models.

自动电路发现的一种有效且可扩展的解决方案是基于优化问题的边缘修剪，该方法在 GPT-2 模型中找到使用少于一半边缘的电路，并与之前的方法在速度和质量上表现出优势。

边缘修剪中的 Transformer 电路寻找

Finding Transformer Circuits with Edge Pruning

Large Language Models (LLMs) have experienced a rapid rise in AI, changing a
wide range of applications with their advanced capabilities. As these models
become increasingly integral to decision-making, the need for thorough
interpretability has never been more critical. Mechanistic Interpretability
offers a pathway to this understanding by identifying and analyzing specific
sub-networks or 'circuits' within these complex systems. A crucial aspect of
this approach is Automated Circuit Discovery, which facilitates the study of
large models like GPT4 or LLAMA in a feasible manner. In this context, our
research evaluates a recent method, Brain-Inspired Modular Training (BIMT),
designed to enhance the interpretability of neural networks. We demonstrate how
BIMT significantly improves the efficiency and quality of Automated Circuit
Discovery, overcoming the limitations of manual methods. Our comparative
analysis further reveals that BIMT outperforms existing models in terms of
circuit quality, discovery time, and sparsity. Additionally, we provide a
comprehensive computational analysis of BIMT, including aspects such as
training duration, memory allocation requirements, and inference speed. This
study advances the larger objective of creating trustworthy and transparent AI
systems in addition to demonstrating how well BIMT works to make neural
networks easier to understand.

研究评估了一种称为 Brain-Inspired Modular Training (BIMT) 的新方法，通过改善神经网络的可解释性，显著提高了自动电路发现的效率和质量，同时超越了现有模型在电路质量、发现时间和稀疏性方面的性能。