We propose a webly-supervised representation learning method that does not
suffer from the annotation unscalability of supervised learning, nor the
computation unscalability of self-supervised learning. Most existing works on
webly-supervised representation learning adopt a vanilla supervised learning
method without accounting for the prevalent noise in the training data, whereas
most prior methods in learning with label noise are less effective for
real-world large-scale noisy data. We propose momentum prototypes (MoPro), a
simple contrastive learning method that achieves online label noise correction,
out-of-distribution sample removal, and representation learning. MoPro achieves
state-of-the-art performance on WebVision, a weakly-labeled noisy dataset.
MoPro also shows superior performance when the pretrained model is transferred
to down-stream image classification and detection tasks. It outperforms the
ImageNet supervised pretrained model by +10.5 on 1-shot classification on VOC,
and outperforms the best self-supervised pretrained model by +17.3 when
finetuned on 1\% of ImageNet labeled samples. Furthermore, MoPro is more robust
to distribution shifts. Code and pretrained models are available at
this https URL

本文提出了一种网络监督下的表示学习方法 —— 动量原型（MoPro），它能够在线修正标签噪声、删除分布之外的样本以及学习表示，实现了最先进的 WebVision 数据集性能，以及更好的图像分类和检测任务性能。