A major challenge in Explainable AI is in correctly interpreting activations
of hidden neurons: accurate interpretations would provide insights into the
question of what a deep learning system has internally detected as relevant on
the input, de-mystifying the otherwise black-box character of deep learning
systems. The state of the art indicates that hidden node activations can, in
some cases, be interpretable in a way that makes sense to humans, but
systematic automated methods that would be able to hypothesize and verify
interpretations of hidden neuron activations are underexplored. In this paper,
we provide such a method and demonstrate that it provides meaningful
interpretations. Our approach is based on using large-scale background
knowledge approximately 2 million classes curated from the Wikipedia concept
hierarchy together with a symbolic reasoning approach called Concept Induction
based on description logics, originally developed for applications in the
Semantic Web field. Our results show that we can automatically attach
meaningful labels from the background knowledge to individual neurons in the
dense layer of a Convolutional Neural Network through a hypothesis and
verification process

解释性人工智能中的一个重要挑战是如何正确解释隐藏神经元的激活：准确的解释可以揭示深度学习系统在输入上检测到的相关信息，解密了深度学习系统的黑盒特性。本文提供了一种基于大规模背景知识和概念归纳的符号推理方法的研究，旨在自动推断和验证隐藏神经元激活的解释，结果表明我们可以自动将背景知识的有意义标签附加到卷积神经网络的密集层的个体神经元上。