The manifold hypothesis posits that high-dimensional data often lies on a
lower-dimensional manifold and that utilizing this manifold as the target space
yields more efficient representations. While numerous traditional
manifold-based techniques exist for dimensionality reduction, their application
in self-supervised learning has witnessed slow progress. The recent MSIMCLR
method combines manifold encoding with SimCLR but requires extremely low target
encoding dimensions to outperform SimCLR, limiting its applicability. This
paper introduces a novel learning paradigm using an unbalanced atlas (UA),
capable of surpassing state-of-the-art self-supervised learning approaches. We
meticulously investigated and engineered the DeepInfomax with an unbalanced
atlas (DIM-UA) method by systematically adapting the Spatiotemporal DeepInfomax
(ST-DIM) framework to align with our proposed UA paradigm, employing rigorous
scientific methodologies throughout the process. The efficacy of DIM-UA is
demonstrated through training and evaluation on the Atari Annotated RAM
Interface (AtariARI) benchmark, a modified version of the Atari 2600 framework
that produces annotated image samples for representation learning. The UA
paradigm improves the existing algorithm significantly when the number of
target encoding dimensions grows. For instance, the mean F1 score averaged over
categories of DIM-UA is ~75% compared to ~70% of ST-DIM when using 16384 hidden
units.

本文提出了一种新的学习方法，利用不平衡的地图来提高自监督学习的效率，并在 AtariARI 基准测试中展示了其比现有算法具有更好的性能。

使用不平衡的图谱进行状态表示学习

State Representation Learning Using an Unbalanced Atlas

Self-supervised learning has made unsupervised pretraining relevant again for
difficult computer vision tasks. The most effective self-supervised methods
involve prediction tasks based on features extracted from diverse views of the
data. DeepInfoMax (DIM) is a self-supervised method which leverages the
internal structure of deep networks to construct such views, forming prediction
tasks between local features which depend on small patches in an image and
global features which depend on the whole image. In this paper, we extend DIM
to the video domain by leveraging similar structure in spatio-temporal
networks, producing a method we call Video Deep InfoMax(VDIM). We find that
drawing views from both natural-rate sequences and temporally-downsampled
sequences yields results on Kinetics-pretrained action recognition tasks which
match or outperform prior state-of-the-art methods that use more costly
large-time-scale transformer models. We also examine the effects of data
augmentation and fine-tuning methods, accomplishingSoTA by a large margin when
training only on the UCF-101 dataset.

本文介绍一种名为 Video Deep InfoMax (VDIM) 的自监督学习方法，将原 DeepInfoMax 扩展至视频领域，其利用内部结构构建视图，并利用这些视图进行预测任务，从而实现对 UCF-101 数据集的行为识别任务的 SoTA 预测结果。