Recent success in training deep neural networks have prompted active
investigation into the features learned on their intermediate layers. Such
research is difficult because it requires making sense of non-linear
computations performed by millions of parameters, but valuable because it
increases our ability to understand current models and create improved versions
of them. In this paper we investigate the extent to which neural networks
exhibit what we call convergent learning, which is when the representations
learned by multiple nets converge to a set of features which are either
individually similar between networks or where subsets of features span similar
low-dimensional spaces. We propose a specific method of probing
representations: training multiple networks and then comparing and contrasting
their individual, learned representations at the level of neurons or groups of
neurons. We begin research into this question using three techniques to
approximately align different neural networks on a feature level: a bipartite
matching approach that makes one-to-one assignments between neurons, a sparse
prediction approach that finds one-to-many mappings, and a spectral clustering
approach that finds many-to-many mappings. This initial investigation reveals a
few previously unknown properties of neural networks, and we argue that future
research into the question of convergent learning will yield many more. The
insights described here include (1) that some features are learned reliably in
multiple networks, yet other features are not consistently learned; (2) that
units learn to span low-dimensional subspaces and, while these subspaces are
common to multiple networks, the specific basis vectors learned are not; (3)
that the representation codes show evidence of being a mix between a local code
and slightly, but not fully, distributed codes across multiple units.

本文研究了神经网络所谓的收敛学习现象，使用多种技术对神经网络进行了实验，发现了一些神经网络的未知属性，包括神经网络中某些特征通常会被成功学习，而其他特征则没有被可靠地学习，单元学习低维子空间，而这些子空间是多个网络共有的，但学习的基向量不一致，表示代码表现出局部代码和略微分布式代码的混合特征。