We present a novel objective function for cluster-based self-supervised
learning (SSL) that is designed to circumvent the triad of failure modes,
namely representation collapse, cluster collapse, and the problem of invariance
to permutations of cluster assignments. This objective consists of three key
components: (i) A generative term that penalizes representation collapse, (ii)
a term that promotes invariance to data augmentations, thereby addressing the
issue of label permutations and (ii) a uniformity term that penalizes cluster
collapse. Additionally, our proposed objective possesses two notable
advantages. Firstly, it can be interpreted from a Bayesian perspective as a
lower bound on the data log-likelihood. Secondly, it enables the training of a
standard backbone architecture without the need for asymmetric elements like
stop gradients, momentum encoders, or specialized clustering layers. Due to its
simplicity and theoretical foundation, our proposed objective is well-suited
for optimization. Experiments on both toy and real world data demonstrate its
effectiveness

我们提出了一种针对基于聚类的自主学习（SSL）的全新客观函数，旨在解决表示坍塌、聚类坍塌和集群分配排列不变性等三种失败模式的问题。我们的目标函数包含三个关键组成部分：（i）惩罚表示坍塌的生成项，（ii）促进对数据增强的不变性，从而解决标签排列问题，以及（ii）惩罚聚类坍塌的一致性项。此外，我们的目标函数具有两个值得注意的优点：首先，从贝叶斯的角度来看，它可视为数据对数似然的下界。其次，它可以训练标准的主干架构，无需使用诸如停止梯度、动量编码器或专门的聚类层等非对称元素。通过其简洁性和理论基础，我们提出的目标函数非常适合优化。在玩具数据和实际数据的实验中证明了其有效性。