In this paper, we propose EqCo (Equivalent Rules for Contrastive Learning) to
make self-supervised learning irrelevant to the number of negative samples in
the contrastive learning framework. Inspired by the InfoMax principle, we point
that the margin term in contrastive loss needs to be adaptively scaled
according to the number of negative pairs in order to keep steady mutual
information bound and gradient magnitude. EqCo bridges the performance gap
among a wide range of negative sample sizes, so that for the first time, we can
use only a few negative pairs (e.g., 16 per query) to perform self-supervised
contrastive training on large-scale vision datasets like ImageNet, while with
almost no accuracy drop. This is quite a contrast to the widely used large
batch training or memory bank mechanism in current practices. Equipped with
EqCo, our simplified MoCo (SiMo) achieves comparable accuracy with MoCov2 on
ImageNet (linear evaluation protocol) while only involves 16 negative pairs per
query instead of 65536, suggesting that large quantities of negative samples is
not a critical factor in contrastive learning frameworks.

本文提出 EqCo（基于等效规则的对比学习），可以使用仅包含极少数量负样本（如每个查询仅包含 16 个）的自监督对比训练在大规模图像数据集（如 ImageNet）上以接近原有准确率进行，缩短了当前实践中大型批量训练或记忆库机制所需的训练时间。此外，作者提出的 simplified MoCo（SiMo）在使用较少负样本情况下在 ImageNet 上表现与 MoCov2 相当，提示负样本数量不是对比学习框架的关键因素。