Self-supervised representation learning~(SSRL) has advanced considerably by
exploiting the transformation invariance assumption under artificially designed
data augmentations. While augmentation-based SSRL algorithms push the
boundaries of performance in computer vision and natural language processing,
they are often not directly applicable to other data modalities, and can
conflict with application-specific data augmentation constraints. This paper
presents an SSRL approach that can be applied to any data modality and network
architecture because it does not rely on augmentations or masking.
Specifically, we show that high-quality data representations can be learned by
reconstructing random data projections. We evaluate the proposed approach on a
wide range of representation learning tasks that span diverse modalities and
real-world applications. We show that it outperforms multiple state-of-the-art
SSRL baselines. Due to its wide applicability and strong empirical results, we
argue that learning from randomness is a fruitful research direction worthy of
attention and further study.

使用无监督学习通过重建随机数据投影来学习高质量数据表示，该方法适用于不同数据模态和网络架构，并超过了多个最先进的无监督学习基线模型的性能。