This paper presents Prototypical Contrastive Learning (PCL), an unsupervised
representation learning method that addresses the fundamental limitations of
instance-wise contrastive learning. PCL not only learns low-level features for
the task of instance discrimination, but more importantly, it implicitly
encodes semantic structures of the data into the learned embedding space.
Specifically, we introduce prototypes as latent variables to help find the
maximum-likelihood estimation of the network parameters in an
Expectation-Maximization framework. We iteratively perform E-step as finding
the distribution of prototypes via clustering and M-step as optimizing the
network via contrastive learning. We propose ProtoNCE loss, a generalized
version of the InfoNCE loss for contrastive learning, which encourages
representations to be closer to their assigned prototypes. PCL outperforms
state-of-the-art instance-wise contrastive learning methods on multiple
benchmarks with substantial improvement in low-resource transfer learning. Code
and pretrained models are available at this https URL

该论文介绍了原型对比学习（PCL）这一无监督表示学习方法，PCL 不仅学习用于实例区分的低级特征，更重要的是它隐含地将数据的语义结构编码到了学习的嵌入空间中，该方法通过增加原型作为潜在变量来寻找最大似然估计，提出了一种基于对比学习的通用损失函数 ProtoNCE loss，此方法在多个基准测试中的表现均优于当前最先进的基于实例对比学习方法，在低资源转移学习方面有了显著的改进。