Understanding what deep network models capture in their learned
representations is a fundamental challenge in computer vision. We present a new
methodology to understanding such vision models, the Visual Concept Connectome
(VCC), which discovers human interpretable concepts and their interlayer
connections in a fully unsupervised manner. Our approach simultaneously reveals
fine-grained concepts at a layer, connection weightings across all layers and
is amendable to global analysis of network structure (e.g., branching pattern
of hierarchical concept assemblies). Previous work yielded ways to extract
interpretable concepts from single layers and examine their impact on
classification, but did not afford multilayer concept analysis across an entire
network architecture. Quantitative and qualitative empirical results show the
effectiveness of VCCs in the domain of image classification. Also, we leverage
VCCs for the application of failure mode debugging to reveal where mistakes
arise in deep networks.

理解深度网络模型在其学习到的表示中捕捉到的内容是计算机视觉中的一个基本挑战。我们提出了一种新的方法来理解这样的视觉模型，即视觉概念连接图（VCC），它以完全无监督的方式发现人类可解释的概念及其在不同层之间的连接。我们的方法同时揭示了网络结构的细粒度概念、所有层之间的连接权重，并且适用于网络结构的全局分析（例如，分层概念组件的分支模式）。此前的工作揭示了如何从单个层提取可解释的概念并检查它们对分类的影响，但未提供在整个网络架构上进行多层概念分析的方法。定量和定性的实证结果显示了 VCC 在图像分类领域的有效性。此外，我们利用 VCC 来进行故障模式调试，以揭示深度网络中错误产生的位置。