In this paper, we propose a genuine group-level contrastive visual
representation learning method whose linear evaluation performance on ImageNet
surpasses the vanilla supervised learning. Two mainstream unsupervised learning
schemes are the instance-level contrastive framework and clustering-based
schemes. The former adopts the extremely fine-grained instance-level
discrimination whose supervisory signal is not efficient due to the false
negatives. Though the latter solves this, they commonly come with some
restrictions affecting the performance. To integrate their advantages, we
design the SMoG method. SMoG follows the framework of contrastive learning but
replaces the contrastive unit from instance to group, mimicking
clustering-based methods. To achieve this, we propose the momentum grouping
scheme which synchronously conducts feature grouping with representation
learning. In this way, SMoG solves the problem of supervisory signal hysteresis
which the clustering-based method usually faces, and reduces the false
negatives of instance contrastive methods. We conduct exhaustive experiments to
show that SMoG works well on both CNN and Transformer backbones. Results prove
that SMoG has surpassed the current SOTA unsupervised representation learning
methods. Moreover, its linear evaluation results surpass the performances
obtained by vanilla supervised learning and the representation can be well
transferred to downstream tasks.

本研究提出了一种名为 SMoG 的基于对比视觉表示学习的方法，其在 ImageNet 数据集上的线性评估表现优于普通的有监督学习方法。该方法结合了实例级对比框架和基于聚类的方法的优点，使用一种名为 momentum grouping 的方案来解决聚类方法通常面临的监督信号滞后问题和实例对比方法存在的误判问题，经全面实验验证可以优于当前的无监督方法并实现线性评估，可应用于下游任务。