Self-Supervised Learning (SSL) methods harness the concept of semantic
invariance by utilizing data augmentation strategies to produce similar
representations for different deformations of the same input. Essentially, the
model captures the shared information among multiple augmented views of
samples, while disregarding the non-shared information that may be beneficial
for downstream tasks. To address this issue, we introduce a module called
CompMod with Meta Comprehensive Regularization (MCR), embedded into existing
self-supervised frameworks, to make the learned representations more
comprehensive. Specifically, we update our proposed model through a bi-level
optimization mechanism, enabling it to capture comprehensive features.
Additionally, guided by the constrained extraction of features using maximum
entropy coding, the self-supervised learning model learns more comprehensive
features on top of learning consistent features. In addition, we provide
theoretical support for our proposed method from information theory and causal
counterfactual perspective. Experimental results show that our method achieves
significant improvement in classification, object detection and instance
segmentation tasks on multiple benchmark datasets.

自我监督学习是通过利用数据增强策略产生相似表示来捕捉多个样本的共享信息，进而学习包含更加全面特征的模型方法。我们提出了一个称为 CompMod with Meta Comprehensive Regularization 的模块，通过双层优化机制来更新模型，使其能够捕捉到更全面的特征。此外，我们还从信息理论和因果反事实的角度为我们提出的方法提供了理论支持。实验证明，我们的方法在多个基准数据集上在分类、目标检测和实例分割任务中取得了显著的改进。