Interpretable representation learning has been playing a key role in creative
intelligent systems. In the music domain, current learning algorithms can
successfully learn various features such as pitch, timbre, chord, texture, etc.
However, most methods rely heavily on music domain knowledge. It remains an
open question what general computational principles give rise to interpretable
representations, especially low-dim factors that agree with human perception.
In this study, we take inspiration from modern physics and use physical
symmetry as a self-consistency constraint for the latent space. Specifically,
it requires the prior model that characterises the dynamics of the latent
states to be equivariant with respect to certain group transformations. We show
that physical symmetry leads the model to learn a linear pitch factor from
unlabelled monophonic music audio in a self-supervised fashion. In addition,
the same methodology can be applied to computer vision, learning a 3D Cartesian
space from videos of a simple moving object without labels. Furthermore,
physical symmetry naturally leads to representation augmentation, a new
technique which improves sample efficiency.

本研究通过利用物理对称性作为潜空间的一致性约束，并将其应用于后续的无监督学习中，以学习诸如线性音高因素之类的低维数据表示，并进一步提出了用于改进样本利用率的表示增强技术。

通过物理对称性学习可解释的低维表示

Learning Interpretable Low-dimensional Representation via Physical Symmetry

We present a lightweight solution to recover 3D pose from multi-view images
captured with spatially calibrated cameras. Building upon recent advances in
interpretable representation learning, we exploit 3D geometry to fuse input
images into a unified latent representation of pose, which is disentangled from
camera view-points. This allows us to reason effectively about 3D pose across
different views without using compute-intensive volumetric grids. Our
architecture then conditions the learned representation on camera projection
operators to produce accurate per-view 2d detections, that can be simply lifted
to 3D via a differentiable Direct Linear Transform (DLT) layer. In order to do
it efficiently, we propose a novel implementation of DLT that is orders of
magnitude faster on GPU architectures than standard SVD-based triangulation
methods. We evaluate our approach on two large-scale human pose datasets (H36M
and Total Capture): our method outperforms or performs comparably to the
state-of-the-art volumetric methods, while, unlike them, yielding real-time
performance.

本文提出了一种基于多视图图像的轻量级解决方案，通过利用新近的可解释表示学习方法，将 3D 几何结构融合到输入图像中，利用摄像机投影运算符对学习表示进行条件化，以产生准确的视角 2D 检测，通过可微的直接线性变换层 (SVD) 将其简单地提升到 3D，并提出一种新方法实现直接线性变换层 (SVD) 以有效地实现三维姿态恢复，同时保证实时性能。