The Symmetric group $S_{n}$ manifests itself in large classes of quantum
systems as the invariance of certain characteristics of a quantum state with
respect to permuting the qubits. The subgroups of $S_{n}$ arise, among many
other contexts, to describe label symmetry of classical images with respect to
spatial transformations, e.g. reflection or rotation. Equipped with the
formalism of geometric quantum machine learning, in this work we propose the
architectures of equivariant quantum convolutional neural networks (EQCNNs)
adherent to $S_{n}$ and its subgroups. We demonstrate that a careful choice of
pixel-to-qubit embedding order can facilitate easy construction of EQCNNs for
small subgroups of $S_{n}$. Our novel EQCNN architecture corresponding to the
full permutation group $S_{n}$ is built by applying all possible QCNNs with
equal probability, which can also be conceptualized as a dropout strategy in
quantum neural networks. For subgroups of $S_{n}$, our numerical results using
MNIST datasets show better classification accuracy than non-equivariant QCNNs.
The $S_{n}$-equivariant QCNN architecture shows significantly improved training
and test performance than non-equivariant QCNN for classification of connected
and non-connected graphs. When trained with sufficiently large number of data,
the $S_{n}$-equivariant QCNN shows better average performance compared to
$S_{n}$-equivariant QNN . These results contribute towards building powerful
quantum machine learning architectures in permutation-symmetric systems.

量子机器学习中，基于置换对称系统的强大算法被提出，利用对称群、等价量子卷积神经网络等关键词建立了置换对称系统的强大算法。

置换等变量量子卷积神经网络

Permutation-equivariant quantum convolutional neural networks

Neural networks that process the parameters of other neural networks find
applications in domains as diverse as classifying implicit neural
representations, generating neural network weights, and predicting
generalization errors. However, existing approaches either overlook the
inherent permutation symmetry in the neural network or rely on intricate
weight-sharing patterns to achieve equivariance, while ignoring the impact of
the network architecture itself. In this work, we propose to represent neural
networks as computational graphs of parameters, which allows us to harness
powerful graph neural networks and transformers that preserve permutation
symmetry. Consequently, our approach enables a single model to encode neural
computational graphs with diverse architectures. We showcase the effectiveness
of our method on a wide range of tasks, including classification and editing of
implicit neural representations, predicting generalization performance, and
learning to optimize, while consistently outperforming state-of-the-art
methods. The source code is open-sourced at
this https URL

本文提出了一种将神经网络表示为参数计算图的方法，借助图神经网络和 transformers 来保持置换对称性，并展示了该方法在多个任务中的有效性。

用于学习神经网络等变表示的图神经网络

Graph Neural Networks for Learning Equivariant Representations of Neural  Networks

The permutation symmetry of neurons in each layer of a deep neural network
gives rise not only to multiple equivalent global minima of the loss function,
but also to first-order saddle points located on the path between the global
minima. In a network of $d-1$ hidden layers with $n_k$ neurons in layers $k =
1, \ldots, d$, we construct smooth paths between equivalent global minima that
lead through a `permutation point' where the input and output weight vectors of
two neurons in the same hidden layer $k$ collide and interchange. We show that
such permutation points are critical points with at least $n_{k+1}$ vanishing
eigenvalues of the Hessian matrix of second derivatives indicating a local
plateau of the loss function. We find that a permutation point for the exchange
of neurons $i$ and $j$ transits into a flat valley (or generally, an extended
plateau of $n_{k+1}$ flat dimensions) that enables all $n_k!$ permutations of
neurons in a given layer $k$ at the same loss value. Moreover, we introduce
high-order permutation points by exploiting the recursive structure in neural
network functions, and find that the number of $K^{\text{th}}$-order
permutation points is at least by a factor
$\sum_{k=1}^{d-1}\frac{1}{2!^K}{n_k-K \choose K}$ larger than the (already
huge) number of equivalent global minima. In two tasks, we illustrate
numerically that some of the permutation points correspond to first-order
saddles (`permutation saddles'): first, in a toy network with a single hidden
layer on a function approximation task and, second, in a multilayer network on
the MNIST task. Our geometric approach yields a lower bound on the number of
critical points generated by weight-space symmetries and provides a simple
intuitive link between previous mathematical results and numerical
observations.

该研究利用深度神经网络计算的几何方法，探讨网络层之间的置换对全局极小化及鞍点问题的影响及其数学意义。

深度网络中的权重空间对称性导致排列鞍点出现，在损失景观中通过等损谷相连

Weight-space symmetry in deep networks gives rise to permutation  saddles, connected by equal-loss valleys across the loss landscape

Skip connections made the training of very deep networks possible and have
become an indispensable component in a variety of neural architectures. A
completely satisfactory explanation for their success remains elusive. Here, we
present a novel explanation for the benefits of skip connections in training
very deep networks. The difficulty of training deep networks is partly due to
the singularities caused by the non-identifiability of the model. Several such
singularities have been identified in previous works: (i) overlap singularities
caused by the permutation symmetry of nodes in a given layer, (ii) elimination
singularities corresponding to the elimination, i.e. consistent deactivation,
of nodes, (iii) singularities generated by the linear dependence of the nodes.
These singularities cause degenerate manifolds in the loss landscape that slow
down learning. We argue that skip connections eliminate these singularities by
breaking the permutation symmetry of nodes, by reducing the possibility of node
elimination and by making the nodes less linearly dependent. Moreover, for
typical initializations, skip connections move the network away from the
"ghosts" of these singularities and sculpt the landscape around them to
alleviate the learning slow-down. These hypotheses are supported by evidence
from simplified models, as well as from experiments with deep networks trained
on real-world datasets.

研究探讨了 skip connection 对深层网络训练中的奇异点造成的影响，证明了 skip connection 可以消除这些奇异点，并在实验中得到了验证。

跳跃连接消除奇异点

Skip Connections Eliminate Singularities

The introduction of convolutional layers greatly advanced the performance of
neural networks on image tasks due to innately capturing a way of encoding and
learning translation-invariant operations, matching one of the underlying
symmetries of the image domain. In comparison, there are a number of problems
in which there are a number of different inputs which are all 'of the same
type' --- multiple particles, multiple agents, multiple stock prices, etc. The
corresponding symmetry to this is permutation symmetry, in that the algorithm
should not depend on the specific ordering of the input data. We discuss a
permutation-invariant neural network layer in analogy to convolutional layers,
and show the ability of this architecture to learn to predict the motion of a
variable number of interacting hard discs in 2D. In the same way that
convolutional layers can generalize to different image sizes, the permutation
layer we describe generalizes to different numbers of objects.

介绍了一种对置换对称的神经网络层，类比于卷积层，可以预测 2D 中密集互动粒子的运动，同时可以适应不同数量的对象。