Neural Persistence is a prominent measure for quantifying neural network
complexity, proposed in the emerging field of topological data analysis in deep
learning. In this work, however, we find both theoretically and empirically
that the variance of network weights and spatial concentration of large weights
are the main factors that impact neural persistence. Whilst this captures
useful information for linear classifiers, we find that no relevant spatial
structure is present in later layers of deep neural networks, making neural
persistence roughly equivalent to the variance of weights. Additionally, the
proposed averaging procedure across layers for deep neural networks does not
consider interaction between layers. Based on our analysis, we propose an
extension of the filtration underlying neural persistence to the whole neural
network instead of single layers, which is equivalent to calculating neural
persistence on one particular matrix. This yields our deep graph persistence
measure, which implicitly incorporates persistent paths through the network and
alleviates variance-related issues through standardisation. Code is available
at this https URL .

神经持续性是在深度学习中的拓扑数据分析领域提出的量化神经网络复杂性的重要指标，本文发现了理论上和实证上对神经持续性产生影响的两个主要因素是网络权重的方差和大权重的空间集中度。然而，我们发现在深度神经网络的后层中不存在相关的空间结构，使得神经持续性与权重的方差大致等价。基于我们的分析，我们提出了将神经持续性的过滤推广到整个神经网络而不是单个层次的方法，这相当于在一个特定矩阵上计算神经持续性。这产生了我们的深度图持续性度量，隐含地融入了网络中的持久路径并通过标准化缓解了与方差相关的问题。代码在此链接可获得。