Unsupervised contrastive learning has achieved outstanding success, while the
mechanism of contrastive loss has been less studied. In this paper, we
concentrate on the understanding of the behaviours of unsupervised contrastive
loss. We will show that the contrastive loss is a hardness-aware loss function,
and the temperature {\tau} controls the strength of penalties on hard negative
samples. The previous study has shown that uniformity is a key property of
contrastive learning. We build relations between the uniformity and the
temperature {\tau} . We will show that uniformity helps the contrastive
learning to learn separable features, however excessive pursuit to the
uniformity makes the contrastive loss not tolerant to semantically similar
samples, which may break the underlying semantic structure and be harmful to
the formation of features useful for downstream tasks. This is caused by the
inherent defect of the instance discrimination objective. Specifically,
instance discrimination objective tries to push all different instances apart,
ignoring the underlying relations between samples. Pushing semantically
consistent samples apart has no positive effect for acquiring a prior
informative to general downstream tasks. A well-designed contrastive loss
should have some extents of tolerance to the closeness of semantically similar
samples. Therefore, we find that the contrastive loss meets a
uniformity-tolerance dilemma, and a good choice of temperature can compromise
these two properties properly to both learn separable features and tolerant to
semantically similar samples, improving the feature qualities and the
downstream performances.

本文研究了无监督对比学习中对比损失的行为特性，发现一种均匀性与相似性之间的权衡关系，通过调节温度参数可以达到学习可分离特征和容错相似样本的效果。