A suitable similarity index for comparing learnt neural networks plays an
important role in understanding the behaviour of the highly-nonlinear
functions, and can provide insights on further theoretical analysis and
empirical studies. We define two key steps when comparing models: firstly, the
representation abstracted from the learnt model, where we propose to leverage
both feature vectors and gradient ones (which are largely ignored in prior
work) into designing the representation of a neural network. Secondly, we
define the employed similarity index which gives desired invariance properties,
and we facilitate the chosen ones with sketching techniques for comparing
various datasets efficiently. Empirically, we show that the proposed approach
provides a state-of-the-art method for computing similarity of neural networks
that are trained independently on different datasets and the tasks defined by
the datasets.

提出了一种新的神经网络相似性计算方法，采用特征向量和梯度向量作为模型表示，结合快速绘图技术可以高效地比较不同数据集中神经网络的相似性。

具有梯度的神经网络的相似性

Similarity of Neural Networks with Gradients

Recent work has sought to understand the behavior of neural networks by
comparing representations between layers and between different trained models.
We examine methods for comparing neural network representations based on
canonical correlation analysis (CCA). We show that CCA belongs to a family of
statistics for measuring multivariate similarity, but that neither CCA nor any
other statistic that is invariant to invertible linear transformation can
measure meaningful similarities between representations of higher dimension
than the number of data points. We introduce a similarity index that measures
the relationship between representational similarity matrices and does not
suffer from this limitation. This similarity index is equivalent to centered
kernel alignment (CKA) and is also closely connected to CCA. Unlike CCA, CKA
can reliably identify correspondences between representations in networks
trained from different initializations.

本文介绍了基于规范相关分析（CCA）方法的神经网络表示比较方法，并提出了一种相似度指数来测量表示相似性矩阵之间的关系，该指数与中心核对齐（CKA）密切相关，但不受高维表示限制，具有可靠性。与 CCA 不同，CKA 方法可在不同初始化的网络表示中可靠地识别对应关系。

神经网络表示的相似性再探讨

Similarity of Neural Network Representations Revisited

Information theory has been taken as a prospective tool for quantifying the
complexity of complex networks. In this paper, we first study the information
entropy or uncertainty of a path using the information theory. Then we apply
the path entropy to the link prediction problem in real-world networks.
Specifically, we propose a new similarity index, namely Path Entropy (PE)
index, which considers the information entropies of shortest paths between node
pairs with penalization to long paths. Empirical experiments demonstrate that
PE index outperforms the mainstream link predictors.

本论文研究了利用信息理论量化复杂网络复杂性的方法，提出了一种基于路径信息熵的相似度指数 —— 路径熵指数，用于解决实际网络中的连接预测问题，实验证明该指数的性能优于主流的连接预测方法。