In light of the recent widespread adoption of AI systems, understanding the
internal information processing of neural networks has become increasingly
critical. Most recently, machine vision has seen remarkable progress by scaling
neural networks to unprecedented levels in dataset and model size. We here ask
whether this extraordinary increase in scale also positively impacts the field
of mechanistic interpretability. In other words, has our understanding of the
inner workings of scaled neural networks improved as well? We here use a
psychophysical paradigm to quantify mechanistic interpretability for a diverse
suite of models and find no scaling effect for interpretability - neither for
model nor dataset size. Specifically, none of the nine investigated
state-of-the-art models are easier to interpret than the GoogLeNet model from
almost a decade ago. Latest-generation vision models appear even less
interpretable than older architectures, hinting at a regression rather than
improvement, with modern models sacrificing interpretability for accuracy.
These results highlight the need for models explicitly designed to be
mechanistically interpretable and the need for more helpful interpretability
methods to increase our understanding of networks at an atomic level. We
release a dataset containing more than 120'000 human responses from our
psychophysical evaluation of 767 units across nine models. This dataset is
meant to facilitate research on automated instead of human-based
interpretability evaluations that can ultimately be leveraged to directly
optimize the mechanistic interpretability of models.

本研究采用心理物理范式研究了多种机器视觉模型的机制可解释性，发现模型的解释性并未因数据集或模型规模的扩大而有所改善，且近代的视觉模型解释性甚至不如近十年前的 GoogLeNet 模型。