Recent advances in self-supervised learning have highlighted the efficacy of data augmentation in learning data representation from unlabeled data. Training a linear model atop these enhanced representations can yield an adept classifier. Despite the remarkable empirical performance, the underlying mechanisms that enable data augmentation to unravel nonlinear data structures into linearly separable representations remain elusive. This paper seeks to bridge this gap by investigating under what conditions learned representations can linearly separate manifolds when data is drawn from a multi-manifold model. Our investigation reveals that data augmentation offers additional information beyond observed data and can thus improve the information-theoretic optimal rate of linear separation capacity. In particular, we show that self-supervised learning can linearly separate manifolds with a smaller distance than unsupervised learning, underscoring the additional benefits of data augmentation. Our theoretical analysis further underscores that the performance of downstream linear classifiers primarily hinges on the linear separability of data representations rather than the size of the labeled data set, reaffirming the viability of constructing efficient classifiers with limited labeled data amid an expansive unlabeled data set.

研究表明，自监督学习和数据增强在从无标签数据中学习数据表示方面具有高效性，通过在这些增强表示之上训练线性模型可以得到熟练的分类器。本文探究了数据增强如何在多流形模型中实现线性分离，发现数据增强提供了超越观察数据的额外信息，可以改善线性分离能力。自监督学习可以通过更小的距离线性分离流形，并强调数据增强的附加好处。理论分析进一步证明下游线性分类器的性能主要取决于数据表示的线性可分性，而不仅仅是标记数据集的大小，验证了在广阔的无标签数据集中利用有限标记数据构建高效分类器的可行性。

自监督表征学习的线性分隔能力