Symmetries built into a neural network have appeared to be very beneficial
for a wide range of tasks as it saves the data to learn them. We depart from
the position that when symmetries are not built into a model a priori, it is
advantageous for robust networks to learn symmetries directly from the data to
fit a task function. In this paper, we present a method to extract symmetries
learned by a neural network and to evaluate the degree to which a network is
invariant to them. With our method, we are able to explicitly retrieve learned
invariances in a form of the generators of corresponding Lie-groups without
prior knowledge of symmetries in the data. We use the proposed method to study
how symmetrical properties depend on a neural network's parameterization and
configuration. We found that the ability of a network to learn symmetries
generalizes over a range of architectures. However, the quality of learned
symmetries depends on the depth and the number of parameters.

提出一种方法来提取神经网络学习的对称性并评估网络对其的不变性程度。结果表明网络的对称性普遍存在于不同的结构中，但学习到的对称性质量取决于深度和参数数量。