Unsupervised learning has recently significantly gained in popularity,
especially with deep learning-based approaches. Despite numerous successes and
approaching supervised-level performance on a variety of academic benchmarks,
it is still hard to train and evaluate SSL models in practice due to the
unsupervised nature of the problem. Even with networks trained in a supervised
fashion, it is often unclear whether they will perform well when transferred to
another domain.
Past works are generally limited to assessing the amount of information
contained in embeddings, which is most relevant for self-supervised learning of
deep neural networks. This works chooses to follow a different approach: can we
quantify how easy it is to linearly separate the data in a stable way? We
survey the literature and uncover three methods that could be potentially used
for evaluating quality of representations. We also introduce one novel method
based on recent advances in understanding the high-dimensional geometric
structure self-supervised learning.
We conduct extensive experiments and study the properties of these metrics
and ones introduced in the previous work. Our results suggest that while there
is no free lunch, there are metrics that can robustly estimate embedding
quality in an unsupervised way.

研究了评估深度神经网络自监督学习中表示质量的多种方法，包括信息嵌入量、线性可分性等，通过实验结果发现了可以无监督评估嵌入质量的方法。

无监督嵌入质量评估

Unsupervised Embedding Quality Evaluation

We present LSD-C, a novel method to identify clusters in an unlabeled
dataset. Our algorithm first establishes pairwise connections in the feature
space between the samples of the minibatch based on a similarity metric. Then
it regroups in clusters the connected samples and enforces a linear separation
between clusters. This is achieved by using the pairwise connections as targets
together with a binary cross-entropy loss on the predictions that the
associated pairs of samples belong to the same cluster. This way, the feature
representation of the network will evolve such that similar samples in this
feature space will belong to the same linearly separated cluster. Our method
draws inspiration from recent semi-supervised learning practice and proposes to
combine our clustering algorithm with self-supervised pretraining and strong
data augmentation. We show that our approach significantly outperforms
competitors on popular public image benchmarks including CIFAR 10/100, STL 10
and MNIST, as well as the document classification dataset Reuters 10K.

LSD-C 是一种新的无标签数据聚类方法，基于相似度度量在特征空间中建立样本之间的成对连接，然后将连接的样本重新分组为聚类，并在聚类之间加强线性分离。该方法借鉴了半监督学习的最新实践，将聚类算法、自监督预训练和强数据增强相结合，证明其在流行的图像基准测试（包括 CIFAR 10/100、STL 10 和 MNIST）以及文档分类数据集 Reuters 10K 上显着优于竞争对手。