Contrastive learning relies on constructing a collection of negative examples
that are sufficiently hard to discriminate against positive queries when their
representations are self-trained. Existing contrastive learning methods either
maintain a queue of negative samples over minibatches while only a small
portion of them are updated in an iteration, or only use the other examples
from the current minibatch as negatives. They could not closely track the
change of the learned representation over iterations by updating the entire
queue as a whole, or discard the useful information from the past minibatches.
Alternatively, we present to directly learn a set of negative adversaries
playing against the self-trained representation. Two players, the
representation network and negative adversaries, are alternately updated to
obtain the most challenging negative examples against which the representation
of positive queries will be trained to discriminate. We further show that the
negative adversaries are updated towards a weighted combination of positive
queries by maximizing the adversarial contrastive loss, thereby allowing them
to closely track the change of representations over time. Experiment results
demonstrate the proposed Adversarial Contrastive (AdCo) model not only achieves
superior performances (a top-1 accuracy of 73.2\% over 200 epochs and 75.7\%
over 800 epochs with linear evaluation on ImageNet), but also can be
pre-trained more efficiently with fewer epochs.

通过直接学习一组对抗的负样本来提高自我学习表征的 对比学习方法，我们的 Adversarial Contrastive (AdCo) 模型得到了更少迭代次数下更出色的性能，线性评估的 Top-1 准确率可以达到 73.2% (200 epochs) 和 75.7% (800 epochs)。