The manifold hypothesis posits that high-dimensional data often lies on a
lower-dimensional manifold and that utilizing this manifold as the target space
yields more efficient representations. While numerous traditional
manifold-based techniques exist for dimensionality reduction, their application
in self-supervised learning has witnessed slow progress. The recent MSIMCLR
method combines manifold encoding with SimCLR but requires extremely low target
encoding dimensions to outperform SimCLR, limiting its applicability. This
paper introduces a novel learning paradigm using an unbalanced atlas (UA),
capable of surpassing state-of-the-art self-supervised learning approaches. We
meticulously investigated and engineered the DeepInfomax with an unbalanced
atlas (DIM-UA) method by systematically adapting the Spatiotemporal DeepInfomax
(ST-DIM) framework to align with our proposed UA paradigm, employing rigorous
scientific methodologies throughout the process. The efficacy of DIM-UA is
demonstrated through training and evaluation on the Atari Annotated RAM
Interface (AtariARI) benchmark, a modified version of the Atari 2600 framework
that produces annotated image samples for representation learning. The UA
paradigm improves the existing algorithm significantly when the number of
target encoding dimensions grows. For instance, the mean F1 score averaged over
categories of DIM-UA is ~75% compared to ~70% of ST-DIM when using 16384 hidden
units.

本文提出了一种新的学习方法，利用不平衡的地图来提高自监督学习的效率，并在 AtariARI 基准测试中展示了其比现有算法具有更好的性能。