Recently, contrastive self-supervised learning, where the proximity of
representations is determined based on the identities of samples, has made
remarkable progress in unsupervised representation learning. SimSiam is a
well-known example in this area, known for its simplicity yet powerful
performance. However, it is known to be sensitive to changes in training
configurations, such as hyperparameters and augmentation settings, due to its
structural characteristics. To address this issue, we focus on the similarity
between contrastive learning and the teacher-student framework in knowledge
distillation. Inspired by the ensemble-based knowledge distillation approach,
the proposed method, EnSiam, aims to improve the contrastive learning procedure
using ensemble representations. This can provide stable pseudo labels,
providing better performance. Experiments demonstrate that EnSiam outperforms
previous state-of-the-art methods in most cases, including the experiments on
ImageNet, which shows that EnSiam is capable of learning high-quality
representations.

本文提出了一种名为 EnSiam 的方法，基于教师 - 学生框架和集成学习思想，使用多个表示来改进对比学习，并提供稳定的伪标签，以提供更好的性能。实验证明，在大多数情况下，EnSiam 能够胜过以往最先进的方法，包括在 ImageNet 上的实验，表明 EnSiam 能够学习到高质量的表示。