Deep neural networks have exhibited remarkable performance across a wide
range of real-world tasks. However, comprehending the underlying reasons for
their effectiveness remains a challenging problem. Interpreting deep neural
networks through examining neurons offers distinct advantages when it comes to
exploring the inner workings of neural networks. Previous research has
indicated that specific neurons within deep vision networks possess semantic
meaning and play pivotal roles in model performance. Nonetheless, the current
methods for generating neuron semantics heavily rely on human intervention,
which hampers their scalability and applicability. To address this limitation,
this paper proposes a novel post-hoc framework for generating semantic
explanations of neurons with large foundation models, without requiring human
intervention or prior knowledge. Our framework is designed to be compatible
with various model architectures and datasets, facilitating automated and
scalable neuron interpretation. Experiments are conducted with both qualitative
and quantitative analysis to verify the effectiveness of our proposed approach.

通过研究神经元来解释深度神经网络并生成语义解释，以解决目前在此方面的限制、缺乏可伸缩性和适用性的问题。

使用大型模型对深度视觉神经元进行自动化自然语言解释

Automated Natural Language Explanation of Deep Visual Neurons with Large  Models

Neuron Interpretation has gained traction in the field of interpretability,
and have provided fine-grained insights into what a model learns and how
language knowledge is distributed amongst its different components. However,
the lack of evaluation benchmark and metrics have led to siloed progress within
these various methods, with very little work comparing them and highlighting
their strengths and weaknesses. The reason for this discrepancy is the
difficulty of creating ground truth datasets, for example, many neurons within
a given model may learn the same phenomena, and hence there may not be one
correct answer. Moreover, a learned phenomenon may spread across several
neurons that work together -- surfacing these to create a gold standard
challenging. In this work, we propose an evaluation framework that measures the
compatibility of a neuron analysis method with other methods. We hypothesize
that the more compatible a method is with the majority of the methods, the more
confident one can be about its performance. We systematically evaluate our
proposed framework and present a comparative analysis of a large set of neuron
interpretation methods. We make the evaluation framework available to the
community. It enables the evaluation of any new method using 20 concepts and
across three pre-trained models.The code is released at
this https URL

该研究提出了一种评估框架，用于比较不同神经元解释方法，通过该框架，可以评估任何新的神经元解释方法，通过与其他大多数方法的兼容性来评估其性能。