Deep neural networks implement a sequence of layer-by-layer operations that
are each relatively easy to understand, but the resulting overall computation
is generally difficult to understand. We consider a simple hypothesis for
interpreting the layer-by-layer construction of useful representations: perhaps
the role of each layer is to reformat information to reduce the "distance" to
the desired outputs. With this framework, the layer-wise computation
implemented by a deep neural network can be viewed as a path through a
high-dimensional representation space. We formalize this intuitive idea of a
"path" by leveraging recent advances in *metric* representational similarity.
We extend existing representational distance methods by computing geodesics,
angles, and projections of representations, going beyond mere layer distances.
We then demonstrate these tools by visualizing and comparing the paths taken by
ResNet and VGG architectures on CIFAR-10. We conclude by sketching additional
ways that this kind of representational geometry can be used to understand and
interpret network training, and to describe novel kinds of similarities between
different models.

深度神经网络的层次计算可以看作是在高维表示空间上的一条路径，通过度量相似性等工具可以理解和解释神经网络的训练过程，并发现不同模型之间的相似之处。