Deep neural networks such as GoogLeNet, ResNet, and BERT have achieved
impressive performance in tasks such as image and text classification. To
understand how such performance is achieved, we probe a trained deep neural
network by studying neuron activations, i.e., combinations of neuron firings,
at various layers of the network in response to a particular input. With a
large number of inputs, we aim to obtain a global view of what neurons detect
by studying their activations. In particular, we develop visualizations that
show the shape of the activation space, the organizational principle behind
neuron activations, and the relationships of these activations within a layer.
Applying tools from topological data analysis, we present TopoAct, a visual
exploration system to study topological summaries of activation vectors. We
present exploration scenarios using TopoAct that provide valuable insights into
learned representations of neural networks. We expect TopoAct to give a
topological perspective that enriches the current toolbox of neural network
analysis, and to provide a basis for network architecture diagnosis and data
anomaly detection.

本文使用拓扑数据分析探究深度神经网络内部层的神经元激活，使用 TopoAct 可视化系统展示获取的激活向量的拓扑信息，为深度神经网络学习表示提供有价值的洞见和可视化工具。