Equivariant neural networks, whose hidden features transform according to
representations of a group G acting on the data, exhibit training efficiency
and an improved generalisation performance. In this work, we extend group
invariant and equivariant representation learning to the field of unsupervised
deep learning. We propose a general learning strategy based on an
encoder-decoder framework in which the latent representation is separated in an
invariant term and an equivariant group action component. The key idea is that
the network learns to encode and decode data to and from a group-invariant
representation by additionally learning to predict the appropriate group action
to align input and output pose to solve the reconstruction task. We derive the
necessary conditions on the equivariant encoder, and we present a construction
valid for any G, both discrete and continuous. We describe explicitly our
construction for rotations, translations and permutations. We test the validity
and the robustness of our approach in a variety of experiments with diverse
data types employing different network architectures.

本文针对深度学习的无监督学习，将群不变和群等变表示学习扩展到了该领域。我们提出了一种基于编码器 - 解码器框架的通用学习策略，其中潜在表示被分为不变项和等变群作用项。在利用预测适当的群作用来对齐输入和输出姿势以解决重建任务时，网络可以学习将数据编码和解码为群不变表示。我们导出依变编码器的必要条件，并针对旋转，平移和置换明确描述了我们的构造。我们在不同网络架构下使用不同数据类型进行各种实验，测试了我们方法的有效性和鲁棒性。

无监督学习群不变和等变表示

Unsupervised Learning of Group Invariant and Equivariant Representations

It is widely believed that engineering a model to be invariant/equivariant
improves generalisation. Despite the growing popularity of this approach, a
precise characterisation of the generalisation benefit is lacking. By
considering the simplest case of linear models, this paper provides the first
provably non-zero improvement in generalisation for invariant/equivariant
models when the target distribution is invariant/equivariant with respect to a
compact group. Moreover, our work reveals an interesting relationship between
generalisation, the number of training examples and properties of the group
action. Our results rest on an observation of the structure of function spaces
under averaging operators which, along with its consequences for feature
averaging, may be of independent interest.

本研究通过研究线性模型，证明了当目标分布与紧致群相关时，使模型不变 / 等变能够改善泛化性能，并揭示了泛化，训练样本数量和群作用属性之间的有趣关系。

等变模型的可证明的严格泛化优势

Provably Strict Generalisation Benefit for Equivariant Models

Using the theory of group action, we first introduce the concept of the
automorphism group of an exponential family or a graphical model, thus
formalizing the general notion of symmetry of a probabilistic model. This
automorphism group provides a precise mathematical framework for lifted
inference in the general exponential family. Its group action partitions the
set of random variables and feature functions into equivalent classes (called
orbits) having identical marginals and expectations. Then the inference problem
is effectively reduced to that of computing marginals or expectations for each
class, thus avoiding the need to deal with each individual variable or feature.
We demonstrate the usefulness of this general framework in lifting two classes
of variational approximation for maximum a posteriori (MAP) inference: local
linear programming (LP) relaxation and local LP relaxation with cycle
constraints; the latter yields the first lifted variational inference algorithm
that operates on a bound tighter than the local constraints.

本文通过引进群作用的概念，对指数族或图模型的自同构群进行形式化定义，提供了一个具有对称性特征的概率模型的精确数学框架，通过群作用将随机变量和特征函数的集合分成具有相同边际分布和期望的等价类（称为轨道），从而将推断问题有效地减少到计算每个等价类的边际分布或期望，避免了处理每个单独的变量或特征的需要。我们演示了这种通用框架在提升两类变分逼近方法中的应用：本地线性规划放松和具有循环约束的本地线性规划放松。后者提供了第一个运行在比本地约束更紧的界面上的提升变分推断算法。