The set of functions parameterized by a linear fully-connected neural network
is a determinantal variety. We investigate the subvariety of functions that are
equivariant or invariant under the action of a permutation group. Examples of
such group actions are translations or $90^\circ$ rotations on images. For such
equivariant or invariant subvarieties, we provide an explicit description of
their dimension, their degree as well as their Euclidean distance degree, and
their singularities. We fully characterize invariance for arbitrary permutation
groups, and equivariance for cyclic groups. We draw conclusions for the
parameterization and the design of equivariant and invariant linear networks,
such as a weight sharing property, and we prove that all invariant linear
functions can be learned by linear autoencoders.

线性全连接神经网络所参数化的函数集合是一个行列式变种。我们研究了在置换群的作用下等变或不变的函数子变种。对于这些等变或不变的子变种，我们提供了其维数、度数以及欧氏距离度数和奇点的明确描述。我们对任意置换群完全表征了不变性和循环群的等变性。我们对等变和不变的线性网络的参数化和设计提出了结论，如权重共享特性，并证明所有不变的线性函数可以通过线性自编码器进行学习。

线性神经网络的几何：对置等变性和不变性的研究

Geometry of Linear Neural Networks: Equivariance and Invariance under  Permutation Groups

Autoencoders are a deep learning model for representation learning. When
trained to minimize the distance between the data and its reconstruction,
linear autoencoders (LAEs) learn the subspace spanned by the top principal
directions but cannot learn the principal directions themselves. In this paper,
we prove that $L_2$-regularized LAEs are symmetric at all critical points and
learn the principal directions as the left singular vectors of the decoder. We
smoothly parameterize the critical manifold and relate the minima to the MAP
estimate of probabilistic PCA. We illustrate these results empirically and
consider implications for PCA algorithms, computational neuroscience, and the
algebraic topology of learning.

证明了 L2 正则化线性自编码器在所有临界点处均对称并学习到解码器的左奇异向量作为主方向，相关结果说明了主成分分析算法、计算神经科学和学习的代数拓扑性质。