Transformer models are increasingly prevalent in various applications, yet our understanding of their internal workings remains limited. This paper investigates the modularity and task specialization of neurons within transformer architectures, focusing on both vision (ViT) and language (Mistral 7B) models. Using a combination of selective pruning and MoEfication clustering techniques, we analyze the overlap and specialization of neurons across different tasks and data subsets. Our findings reveal evidence of task-specific neuron clusters, with varying degrees of overlap between related tasks. We observe that neuron importance patterns persist to some extent even in randomly initialized models, suggesting an inherent structure that training refines. Additionally, we find that neuron clusters identified through MoEfication correspond more strongly to task-specific neurons in earlier and later layers of the models. This work contributes to a more nuanced understanding of transformer internals and offers insights into potential avenues for improving model interpretability and efficiency.

本文探讨了变换器结构中神经元的模块化和任务专业化，旨在填补对变换器内部机制的理解空白。通过选择性裁剪和MoEfication聚类技术，分析了神经元在不同任务和数据子集中的重叠和专业化情况，发现存在任务特定的神经元簇，这为提升模型的可解释性和效率提供了新的视角。

变换器中的模块化：神经元可分离性与专业化研究