Measuring similarity of neural networks has become an issue of great importance and research interest to understand and utilize differences of neural networks. While there are several perspectives on how neural networks can be similar, we specifically focus on two complementing perspectives, i.e., (i) representational similarity, which considers how activations of intermediate neural layers differ, and (ii) functional similarity, which considers how models differ in their outputs. In this survey, we provide a comprehensive overview of these two families of similarity measures for neural network models. In addition to providing detailed descriptions of existing measures, we summarize and discuss results on the properties and relationships of these measures, and point to open research problems. Further, we provide practical recommendations that can guide researchers as well as practitioners in applying the measures. We hope our work lays a foundation for our community to engage in more systematic research on the properties, nature and applicability of similarity measures for neural network models.

该论文综述了神经网络模型的两种相似性度量方法，即表示相似度和功能相似度，介绍了现有度量方法的详细描述，并总结和讨论了这些度量方法的特性和关系，并指出了开放的研究问题和为研究人员及从业者提供实用的建议。

神经网络模型的相似性：功能和表征度量的调查