Symmetry has been a fundamental tool in the exploration of a broad range of complex systems. In machine learning, symmetry has been explored in both models and data. In this paper we seek to connect the symmetries arising from the architecture of a family of models with the symmetries of that family's internal representation of data. We do this by calculating a set of fundamental symmetry groups, which we call the \emph{intertwiner groups} of the model. Each of these arises from a particular nonlinear layer of the model and different nonlinearities result in different symmetry groups. These groups change the weights of a model in such a way that the underlying function that the model represents remains constant but the internal representations of data inside the model may change. We connect intertwiner groups to a model's internal representations of data through a range of experiments that probe similarities between hidden states across models with the same architecture. Our work suggests that the symmetries of a network are propagated into the symmetries in that network's representation of data, providing us with a better understanding of how architecture affects the learning and prediction process. Finally, we speculate that for ReLU networks, the intertwiner groups may provide a justification for the common practice of concentrating model interpretability exploration on the activation basis in hidden layers rather than arbitrary linear combinations thereof.

对于机器学习模型的对称性和内部数据表示之间的关联，我们提出了本文中阐述的模型内联群的概念，通过相似实验，我们将内联群与具有相同架构的模型之间的隐藏状态相似性联系起来，从而更好地理解体系结构如何影响学习和预测过程。最后，我们猜测对于ReLU网络，内联群可能提供一种合理的解释为什么要在隐藏层中集中探索模型可解释性。

深度学习模型及其内部表示的对称性