Neural Persistence is a prominent measure for quantifying neural network
complexity, proposed in the emerging field of topological data analysis in deep
learning. In this work, however, we find both theoretically and empirically
that the variance of network weights and spatial concentration of large weights
are the main factors that impact neural persistence. Whilst this captures
useful information for linear classifiers, we find that no relevant spatial
structure is present in later layers of deep neural networks, making neural
persistence roughly equivalent to the variance of weights. Additionally, the
proposed averaging procedure across layers for deep neural networks does not
consider interaction between layers. Based on our analysis, we propose an
extension of the filtration underlying neural persistence to the whole neural
network instead of single layers, which is equivalent to calculating neural
persistence on one particular matrix. This yields our deep graph persistence
measure, which implicitly incorporates persistent paths through the network and
alleviates variance-related issues through standardisation. Code is available
at this https URL .

神经持续性是在深度学习中的拓扑数据分析领域提出的量化神经网络复杂性的重要指标，本文发现了理论上和实证上对神经持续性产生影响的两个主要因素是网络权重的方差和大权重的空间集中度。然而，我们发现在深度神经网络的后层中不存在相关的空间结构，使得神经持续性与权重的方差大致等价。基于我们的分析，我们提出了将神经持续性的过滤推广到整个神经网络而不是单个层次的方法，这相当于在一个特定矩阵上计算神经持续性。这产生了我们的深度图持续性度量，隐含地融入了网络中的持久路径并通过标准化缓解了与方差相关的问题。代码在此链接可获得。

深度图像持久性解决神经持久性的注意事项

Addressing caveats of neural persistence with deep graph persistence

While many approaches to make neural networks more fathomable have been
proposed, they are restricted to interrogating the network with input data.
Measures for characterizing and monitoring structural properties, however, have
not been developed. In this work, we propose neural persistence, a complexity
measure for neural network architectures based on topological data analysis on
weighted stratified graphs. To demonstrate the usefulness of our approach, we
show that neural persistence reflects best practices developed in the deep
learning community such as dropout and batch normalization. Moreover, we derive
a neural persistence-based stopping criterion that shortens the training
process while achieving comparable accuracies as early stopping based on
validation loss.

本文提出了一种基于拓扑数据分析的神经网络架构复杂性度量 —— 神经持久性，用于表征和监控结构特性，并通过此方法展示了 dropout 和 batch normalization 等深度学习社区开发的最佳实践。此外，还提出了一种基于神经持久性停止准则，可在缩短训练过程同时实现与基于验证损失的早停相当的准确性。