Recent advances in contrastive learning have enlightened diverse applications
across various semi-supervised fields. Jointly training supervised learning and
unsupervised learning with a shared feature encoder becomes a common scheme.
Though it benefits from taking advantage of both feature-dependent information
from self-supervised learning and label-dependent information from supervised
learning, this scheme remains suffering from bias of the classifier. In this
work, we systematically explore the relationship between self-supervised
learning and supervised learning, and study how self-supervised learning helps
robust data-efficient deep learning. We propose hyperspherical consistency
regularization (HCR), a simple yet effective plug-and-play method, to
regularize the classifier using feature-dependent information and thus avoid
bias from labels. Specifically, HCR first projects logits from the classifier
and feature projections from the projection head on the respective hypersphere,
then it enforces data points on hyperspheres to have similar structures by
minimizing binary cross entropy of pairwise distances' similarity metrics.
Extensive experiments on semi-supervised and weakly-supervised learning
demonstrate the effectiveness of our method, by showing superior performance
with HCR.

本文介绍了一种基于共享特征编码器的监督学习和无监督学习的联合训练方式，结合自监督学习和有标签信息可以提高性能，但该方案仍存在分类器偏差问题。因此，提出了一种名为超球面一致性正则化 (HCR) 的简单但高效的方法来避免标签偏差问题，通过特征依赖信息对分类器进行正则化。该方法通过最小化二元交叉熵的成对距离相似性度量来实现数据点具有相似结构，实验证明该方法在半监督和弱监督学习中表现优异。