Self-Supervised Learning (SSL) surmises that inputs and pairwise positive
relationships are enough to learn meaningful representations. Although SSL has
recently reached a milestone: outperforming supervised methods in many
modalities\dots the theoretical foundations are limited, method-specific, and
fail to provide principled design guidelines to practitioners. In this paper,
we propose a unifying framework under the helm of spectral manifold learning to
address those limitations. Through the course of this study, we will rigorously
demonstrate that VICReg, SimCLR, BarlowTwins et al. correspond to eponymous
spectral methods such as Laplacian Eigenmaps, Multidimensional Scaling et al.
This unification will then allow us to obtain (i) the closed-form optimal
representation for each method, (ii) the closed-form optimal network parameters
in the linear regime for each method, (iii) the impact of the pairwise
relations used during training on each of those quantities and on downstream
task performances, and most importantly, (iv) the first theoretical bridge
between contrastive and non-contrastive methods towards global and local
spectral embedding methods respectively, hinting at the benefits and
limitations of each. For example, (i) if the pairwise relation is aligned with
the downstream task, any SSL method can be employed successfully and will
recover the supervised method, but in the low data regime, VICReg's invariance
hyper-parameter should be high; (ii) if the pairwise relation is misaligned
with the downstream task, VICReg with small invariance hyper-parameter should
be preferred over SimCLR or BarlowTwins.

本文提出了一种基于谱流形学的统一框架，以解决自监督学习方法的局限性并提供有意义的表示学习方法，通过将 VICReg，SimCLR 和 BarlowTwins 等自监督学习方法与谱方法相对应，得到了闭合形式的最佳表示以及线性区间内的网络最佳参数，并揭示了对训练中使用的成对关系以及下游任务性能的影响，以及对于两种谱嵌入方法之间的第一种理论桥梁的暗示。