Transformers demonstrate impressive performance on a range of reasoning benchmarks. To evaluate the degree to which these abilities are a result of actual reasoning, existing work has focused on developing sophisticated benchmarks for behavioral studies. However, these studies do not provide insights into the internal mechanisms driving the observed capabilities. To improve our understanding of the internal mechanisms of transformers, we present a comprehensive mechanistic analysis of a transformer trained on a synthetic reasoning task. We identify a set of interpretable mechanisms the model uses to solve the task, and validate our findings using correlational and causal evidence. Our results suggest that it implements a depth-bounded recurrent mechanisms that operates in parallel and stores intermediate results in selected token positions. We anticipate that the motifs we identified in our synthetic setting can provide valuable insights into the broader operating principles of transformers and thus provide a basis for understanding more complex models.

通过对合成推理任务进行综合机械分析，我们鉴定了一组可解释的机制，这个模型用来解决任务，并使用相关和因果证据验证了我们的发现。我们的结果表明，它实现了一组深度有限的并行循环机制，并将中间结果存储在选择的令牌位置，我们期望我们在合成环境中鉴定的这些模式可以为理解变压器的更广泛操作原理提供有价值的见解。

一个关于训练于符号多步推理任务的Transformer的机制分析