In standard supervised machine learning, it is necessary to provide a label
for every input in the data. While raw data in many application domains is
easily obtainable on the Internet, manual labelling of this data is
prohibitively expensive. To circumvent this issue, contrastive learning methods
produce low-dimensional vector representations (also called features) of
high-dimensional inputs on large unlabelled datasets. This is done by training
with a contrastive loss function, which enforces that similar inputs have high
inner product and dissimilar inputs have low inner product in the feature
space. Rather than annotating each input individually, it suffices to define a
means of sampling pairs of similar and dissimilar inputs. Contrastive features
can then be fed as inputs to supervised learning systems on much smaller
labelled datasets to obtain high accuracy on end tasks of interest.
The goal of this thesis is to provide an overview of the current theoretical
understanding of contrastive learning, specifically as it pertains to the
minimizers of contrastive loss functions and their relationship to prior
methods for learning features from unlabelled data. We highlight popular
contrastive loss functions whose minimizers implicitly approximate a positive
semidefinite (PSD) kernel. The latter is a well-studied object in functional
analysis and learning theory that formalizes a notion of similarity between
elements of a space. PSD kernels provide an implicit definition of features
through the theory of reproducing kernel Hilbert spaces.

通过对无标签数据进行对比学习，可以生成低维的特征向量表示，这些特征向量可以作为输入来提高有标签数据上的监督学习系统的准确性，并探讨了对比损失函数的最小化器及其与以往学习无标签数据方法的关系。