We employ a toolset -- dubbed Dr. Frankenstein -- to analyse the similarity of representations in deep neural networks. With this toolset, we aim to match the activations on given layers of two trained neural networks by joining them with a stitching layer. We demonstrate that the inner representations emerging in deep convolutional neural networks with the same architecture but different initializations can be matched with a surprisingly high degree of accuracy even with a single, affine stitching layer. We choose the stitching layer from several possible classes of linear transformations and investigate their performance and properties. The task of matching representations is closely related to notions of similarity. Using this toolset, we also provide a novel viewpoint on the current line of research regarding similarity indices of neural network representations: the perspective of the performance on a task.

本文通过利用Dr. Frankenstein工具集来分析深度神经网络的相似性，尝试匹配由两个已经训练好的模型中给定层的激活，通过使用一个拼接层将它们合并起来。我们证明了即使只有一个单一的仿射拼接层，具有相同结构但不同初始化的卷积神经网络中出现的内部表示仍然可以以令人惊讶的高精度进行匹配，同时我们也从任务的角度提供了关于神经网络相似性指数的新视角。

神经网络表示的相似性和匹配