The set of functions parameterized by a linear fully-connected neural network
is a determinantal variety. We investigate the subvariety of functions that are
equivariant or invariant under the action of a permutation group. Examples of
such group actions are translations or $90^\circ$ rotations on images. For such
equivariant or invariant subvarieties, we provide an explicit description of
their dimension, their degree as well as their Euclidean distance degree, and
their singularities. We fully characterize invariance for arbitrary permutation
groups, and equivariance for cyclic groups. We draw conclusions for the
parameterization and the design of equivariant and invariant linear networks,
such as a weight sharing property, and we prove that all invariant linear
functions can be learned by linear autoencoders.

线性全连接神经网络所参数化的函数集合是一个行列式变种。我们研究了在置换群的作用下等变或不变的函数子变种。对于这些等变或不变的子变种，我们提供了其维数、度数以及欧氏距离度数和奇点的明确描述。我们对任意置换群完全表征了不变性和循环群的等变性。我们对等变和不变的线性网络的参数化和设计提出了结论，如权重共享特性，并证明所有不变的线性函数可以通过线性自编码器进行学习。

线性神经网络的几何：对置等变性和不变性的研究

Geometry of Linear Neural Networks: Equivariance and Invariance under  Permutation Groups

We consider the problem of discovering subgroup $H$ of permutation group
$S_{n}$. Unlike the traditional $H$-invariant networks wherein $H$ is assumed
to be known, we present a method to discover the underlying subgroup, given
that it satisfies certain conditions. Our results show that one could discover
any subgroup of type $S_{k} (k \leq n)$ by learning an $S_{n}$-invariant
function and a linear transformation. We also prove similar results for cyclic
and dihedral subgroups. Finally, we provide a general theorem that can be
extended to discover other subgroups of $S_{n}$. We also demonstrate the
applicability of our results through numerical experiments on image-digit sum
and symmetric polynomial regression tasks.

我们考虑寻找置换群 S_n 的子群 H 的问题。我们提出了一种方法，可以在满足一定条件的情况下发现隐藏的子群 H。我们的结果表明通过学习一个 S_n 不变函数和一个线性变换可以发现任何类型为 S_k (k ≤ n) 的子群。我们还证明了循环子群和二面体子群的类似结果。最后，我们提供了一个可以扩展到发现 S_n 的其他子群的通用定理。我们还通过图像位数和对称多项式回归任务的数值实验验证了我们的结果的适用性。