Although deep neural networks have shown well-performance in various tasks,
the poor interpretability of the models is always criticized. In the paper, we
propose a new interpretable neural network method, by embedding neurons into
the semantic space to extract their intrinsic global semantics. In contrast to
previous methods that probe latent knowledge inside the model, the proposed
semantic vector externalizes the latent knowledge to static knowledge, which is
easy to exploit. Specifically, we assume that neurons with similar activation
are of similar semantic information. Afterwards, semantic vectors are optimized
by continuously aligning activation similarity and semantic vector similarity
during the training of the neural network. The visualization of semantic
vectors allows for a qualitative explanation of the neural network. Moreover,
we assess the static knowledge quantitatively by knowledge distillation tasks.
Empirical experiments of visualization show that semantic vectors describe
neuron activation semantics well. Without the sample-by-sample guidance from
the teacher model, static knowledge distillation exhibit comparable or even
superior performance with existing relation-based knowledge distillation
methods.

通过将神经元嵌入语义空间来提取它们内在的全局语义，提出了一种新的可解释神经网络方法。神经元的激活相似性和语义向量相似性被不断对齐来优化语义向量，可视化语义向量可以定性地解释神经网络的工作机制，并经过了定量的知识蒸馏任务的实验验证，表明提出的静态知识蒸馏方法表现出了可比或甚至优于现有的基于关系的知识蒸馏方法。