Circuit analysis of any certain model behavior is a central task in
mechanistic interpretability. We introduce our circuit discovery pipeline with
sparse autoencoders (SAEs) and a variant called skip SAEs. With these two
modules inserted into the model, the model's computation graph with respect to
OV and MLP circuits becomes strictly linear. Our methods do not require linear
approximation to compute the causal effect of each node. This fine-grained
graph enables identifying both end-to-end and local circuits accounting for
either logits or intermediate features. We can scalably apply this pipeline
with a technique called Hierarchical Attribution. We analyze three kind of
circuits in GPT2-Small, namely bracket, induction and Indirect Object
Identification circuits. Our results reveal new findings underlying existing
discoveries.

采用稀疏自编码器（SAEs）和跳跃 SAEs 为基础，引入电路发现流程，使用 Hierarchical Attribution 方法对于 GPT2-Small 模型分析了三种电路类型（括号电路、归纳电路和间接对象识别电路），揭示了现有发现之下的新发现。

使用线性计算图自动识别局部和全局电路

Automatically Identifying Local and Global Circuits with Linear  Computation Graphs

The black-box nature of deep learning models in NLP hinders their widespread
application. The research focus has shifted to Hierarchical Attribution (HA)
for its ability to model feature interactions. Recent works model
non-contiguous combinations with a time-costly greedy search in Eculidean
spaces, neglecting underlying linguistic information in feature
representations. In this work, we introduce a novel method, namely Poincar\'e
Explanation (PE), for modeling feature interactions using hyperbolic spaces in
an $O(n^2logn)$ time complexity. Inspired by Poincar\'e model, we propose a
framework to project the embeddings into hyperbolic spaces, which exhibit
better inductive biases for syntax and semantic hierarchical structures.
Eventually, we prove that the hierarchical clustering process in the projected
space could be viewed as building a minimum spanning tree and propose a time
efficient algorithm. Experimental results demonstrate the effectiveness of our
approach.

该研究介绍了一种称为 Poincaré 解释（PE）的新方法，在 O (n^2logn) 时间复杂度内使用双曲空间建模特征交互，并证明了在投影空间中的层次聚类过程可以视为构建最小生成树，并提出了一种时间高效的算法。实验结果证明了我们方法的有效性。

PE：一种用于快速文本层次生成的 Poincare 解释方法

PE: A Poincare Explanation Method for Fast Text Hierarchy Generation

The opaqueness of deep NLP models has motivated the development of methods
for interpreting how deep models predict. Recently, work has introduced
hierarchical attribution, which produces a hierarchical clustering of words,
along with an attribution score for each cluster. However, existing work on
hierarchical attribution all follows the connecting rule, limiting the cluster
to a continuous span in the input text. We argue that the connecting rule as an
additional prior may undermine the ability to reflect the model decision
process faithfully. To this end, we propose to generate hierarchical
explanations without the connecting rule and introduce a framework for
generating hierarchical clusters. Experimental results and further analysis
show the effectiveness of the proposed method in providing high-quality
explanations for reflecting model predicting process.

本文提出了一种不需要连接规则的层次化解释生成框架，可以用于生成单词的层次式聚类，以提供高质量的解释，并反映模型决策过程。