Long-tailed distributions frequently emerge in real-world data, where a large
number of minority categories contain a limited number of samples. Such
imbalance issue considerably impairs the performance of standard supervised
learning algorithms, which are mainly designed for balanced training sets.
Recent investigations have revealed that supervised contrastive learning
exhibits promising potential in alleviating the data imbalance. However, the
performance of supervised contrastive learning is plagued by an inherent
challenge: it necessitates sufficiently large batches of training data to
construct contrastive pairs that cover all categories, yet this requirement is
difficult to meet in the context of class-imbalanced data. To overcome this
obstacle, we propose a novel probabilistic contrastive (ProCo) learning
algorithm that estimates the data distribution of the samples from each class
in the feature space, and samples contrastive pairs accordingly. In fact,
estimating the distributions of all classes using features in a small batch,
particularly for imbalanced data, is not feasible. Our key idea is to introduce
a reasonable and simple assumption that the normalized features in contrastive
learning follow a mixture of von Mises-Fisher (vMF) distributions on unit
space, which brings two-fold benefits. First, the distribution parameters can
be estimated using only the first sample moment, which can be efficiently
computed in an online manner across different batches. Second, based on the
estimated distribution, the vMF distribution allows us to sample an infinite
number of contrastive pairs and derive a closed form of the expected
contrastive loss for efficient optimization. Our code is available at
this https URL

在研究中，我们提出了一种新颖的概率对比学习算法（ProCo），它通过估计特征空间中每个类别的样本数据分布并相应地采样对比对，以克服由于数据不平衡而导致的标准有监督学习算法性能下降的难题。我们引入了一个合理简单的假设，即在对比学习中，归一化特征遵循单位空间上的一组 von Mises-Fisher（vMF）分布的混合分布，利用该分布参数的估计，我们可以采样无限数量的对比对，并得到了对比损失的闭合形式，从而实现高效优化。