The goal of generality in machine learning is to achieve excellent
performance on various unseen tasks and domains. Recently, self-supervised
learning (SSL) has been regarded as an effective method to achieve this goal.
It can learn high-quality representations from unlabeled data and achieve
promising empirical performance on multiple downstream tasks. Existing SSL
methods mainly constrain generality from two aspects: (i) large-scale training
data, and (ii) learning task-level shared knowledge. However, these methods
lack explicit modeling of the SSL generality in the learning objective, and the
theoretical understanding of SSL's generality remains limited. This may cause
SSL models to overfit in data-scarce situations and generalize poorly in the
real world, making it difficult to achieve true generality. To address these
issues, we provide a theoretical definition of generality in SSL and define a
$\sigma$-measurement to help quantify it. Based on this insight, we explicitly
model generality into self-supervised learning and further propose a novel SSL
framework, called GeSSL. It introduces a self-motivated target based on
$\sigma$-measurement, which enables the model to find the optimal update
direction towards generality. Extensive theoretical and empirical evaluations
demonstrate the superior performance of the proposed GeSSL.

自我监督学习的广义性理论定义及新型 GeSSL 框架的提出，能够在数据稀缺和真实世界中实现优越性能。