Unsupervised and self-supervised representation learning has become popular
in recent years for learning useful features from unlabelled data.
Representation learning has been mostly developed in the neural network
literature, and other models for representation learning are surprisingly
unexplored. In this work, we introduce and analyze several kernel-based
representation learning approaches: Firstly, we define two kernel
Self-Supervised Learning (SSL) models using contrastive loss functions and
secondly, a Kernel Autoencoder (AE) model based on the idea of embedding and
reconstructing data. We argue that the classical representer theorems for
supervised kernel machines are not always applicable for (self-supervised)
representation learning, and present new representer theorems, which show that
the representations learned by our kernel models can be expressed in terms of
kernel matrices. We further derive generalisation error bounds for
representation learning with kernel SSL and AE, and empirically evaluate the
performance of these methods in both small data regimes as well as in
comparison with neural network based models.

使用基于核函数的自监督学习和自编码器模型进行表示学习，引入新的表示定理并通过实验评估了这些方法在小数据环境和与神经网络模型的比较中的性能。

核方法的非参数表示学习

Non-Parametric Representation Learning with Kernels

Kernel-based models such as kernel ridge regression and Gaussian processes
are ubiquitous in machine learning applications for regression and
optimization. It is well known that a major downside for kernel-based models is
the high computational cost; given a dataset of $n$ samples, the cost grows as
$\mathcal{O}(n^3)$. Existing sparse approximation methods can yield a
significant reduction in the computational cost, effectively reducing the
actual cost down to as low as $\mathcal{O}(n)$ in certain cases. Despite this
remarkable empirical success, significant gaps remain in the existing results
for the analytical bounds on the error due to approximation. In this work, we
provide novel confidence intervals for the Nystr\"om method and the sparse
variational Gaussian process approximation method, which we establish using
novel interpretations of the approximate (surrogate) posterior variance of the
models. Our confidence intervals lead to improved performance bounds in both
regression and optimization problems.

本文针对核方法的高计算成本问题，提出了 Nyström 方法和稀疏变分高斯过程逼近方法的置信区间，从而改进了模型在回归和优化问题中的性能界限。