Contrastive learning, especially Self-Supervised Contrastive Learning (SSCL),
has achieved great success in extracting powerful features from unlabeled data,
enabling comparable performance to the supervised counterpart. In this work, we
contribute to the theoretical understanding of SSCL and uncover its connection
to the classic data visualization method, Stochastic Neighbor Embedding (SNE).
In the perspective of SNE, whose goal is matching pairwise distance, SSCL can
be viewed as a special case with the input space pairwise distance specified by
constructed "positive" pairs from data augmentation. The established
correspondence facilitates deeper theoretical understandings of learned
features of SSCL, as well as methodological guidelines for practical
improvement. Specifically, through the lens of SNE, not only can we re-derive
the alignment and uniformity principle, but also provide novel analysis on
domain-agnostic augmentations and implicit bias. To illustrate the practical
advantage, we demonstrate that the modifications from SNE to $t$-SNE can also
be adopted in the SSCL setting, achieving significant improvement in both
in-distribution and out-of-distribution generalization.

本文探讨了无监督自学习对偶损失模型（SSCL）的理论基础，提出了在随机邻居嵌入（SNE）视角下，使用数据增强构建 “正” 偶对实现了对输入空间对比度的优化，通过与 SNE 对应，进一步提供了方法学指导并证明了使用域不可知增强与隐式偏见得到的潜在优势，同时证明了从 SNE 到 t-SNE 转换的适用性并在实验中取得了相应的优化效果。

对比学习暗中进行着随机邻居嵌入

Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding

Extensions and variants are given for the well-known comparison principle for
Gaussian processes based on ordering by pairwise distance.

介绍了基于成对距离排序的高斯过程比较原理的扩展和变体。