Automated interpretability research has recently attracted attention as a
potential research direction that could scale explanations of neural network
behavior to large models. Existing automated circuit discovery work applies
activation patching to identify subnetworks responsible for solving specific
tasks (circuits). In this work, we show that a simple method based on
attribution patching outperforms all existing methods while requiring just two
forward passes and a backward pass. We apply a linear approximation to
activation patching to estimate the importance of each edge in the
computational subgraph. Using this approximation, we prune the least important
edges of the network. We survey the performance and limitations of this method,
finding that averaged over all tasks our method has greater AUC from circuit
recovery than other methods.

通过应用基于归因修补的简单方法来剔除神经网络中最不重要的边缘，我们的研究比现有方法在电路恢复方面具有更高的 AUC。

归因修补优于自动电路发现

Attribution Patching Outperforms Automated Circuit Discovery

In this technical report, we explore the behavior of Recursive Feature
Machines (RFMs), a type of novel kernel machine that recursively learns
features via the average gradient outer product, through a series of
experiments on regression datasets. When successively adding random noise
features to a dataset, we observe intriguing patterns in the Mean Squared Error
(MSE) curves with the test MSE exhibiting a decrease-increase-decrease pattern.
This behavior is consistent across different dataset sizes, noise parameters,
and target functions. Interestingly, the observed MSE curves show similarities
to the "double descent" phenomenon observed in deep neural networks, hinting at
new connection between RFMs and neural network behavior. This report lays the
groundwork for future research into this peculiar behavior.

本研究通过对回归数据集的一系列实验研究，探讨了一种新型核机器 —— 递归特征机的行为，发现与神经网络行为类似的双下降现象表现，揭示了 RFMs 和神经网络行为之间的新联系，为未来的研究打下了基础。

递归特征机器的特征缩放

On Feature Scaling of Recursive Feature Machines

To understand neural network behavior, recent works quantitatively compare
different networks' learned representations using canonical correlation
analysis (CCA), centered kernel alignment (CKA), and other dissimilarity
measures. Unfortunately, these widely used measures often disagree on
fundamental observations, such as whether deep networks differing only in
random initialization learn similar representations. These disagreements raise
the question: which, if any, of these dissimilarity measures should we believe?
We provide a framework to ground this question through a concrete test:
measures should have sensitivity to changes that affect functional behavior,
and specificity against changes that do not. We quantify this through a variety
of functional behaviors including probing accuracy and robustness to
distribution shift, and examine changes such as varying random initialization
and deleting principal components. We find that current metrics exhibit
different weaknesses, note that a classical baseline performs surprisingly
well, and highlight settings where all metrics appear to fail, thus providing a
challenge set for further improvement.

本文提供了一个框架来验证神经网络的行为，通过功能行为敏感性和特异性等方面测试常用的神经网络模型评估方法，研究发现当前的评估指标存在不同缺陷，一个经典的基线表现出人意料的好，作者对所有指标都无法胜任的场景进行了强调，并为未来的研究提供了挑战性的基础数据。

利用统计检验来建立表示相似性

Grounding Representation Similarity with Statistical Testing

Recent work has shown great promise in explaining neural network behavior. In
particular, feature attribution methods explain which features were most
important to a model's prediction on a given input. However, for many tasks,
simply knowing which features were important to a model's prediction may not
provide enough insight to understand model behavior. The interactions between
features within the model may better help us understand not only the model, but
also why certain features are more important than others. In this work, we
present Integrated Hessians, an extension of Integrated Gradients that explains
pairwise feature interactions in neural networks. Integrated Hessians overcomes
several theoretical limitations of previous methods to explain interactions,
and unlike such previous methods is not limited to a specific architecture or
class of neural network. Additionally, we find that our method is faster than
existing methods when the number of features is large, and outperforms previous
methods on existing quantitative benchmarks. Code available at
this https URL

本研究介绍了 Integrated Hessians，这是 Integrated Gradients 的扩展，用于解释神经网络中的成对特征交互；与以前的方法相比，Integrated Hessians 具有更快的速度和更好的性能，适用于各种神经网络架构或类别。