The Sliced Wasserstein (SW) distance has become a popular alternative to the
Wasserstein distance for comparing probability measures. Widespread
applications include image processing, domain adaptation and generative
modelling, where it is common to optimise some parameters in order to minimise
SW, which serves as a loss function between discrete probability measures
(since measures admitting densities are numerically unattainable). All these
optimisation problems bear the same sub-problem, which is minimising the Sliced
Wasserstein energy. In this paper we study the properties of $\mathcal{E}: Y
\longmapsto \mathrm{SW}_2^2(\gamma_Y, \gamma_Z)$, i.e. the SW distance between
two uniform discrete measures with the same amount of points as a function of
the support $Y \in \mathbb{R}^{n \times d}$ of one of the measures. We
investigate the regularity and optimisation properties of this energy, as well
as its Monte-Carlo approximation $\mathcal{E}_p$ (estimating the expectation in
SW using only $p$ samples) and show convergence results on the critical points
of $\mathcal{E}_p$ to those of $\mathcal{E}$, as well as an almost-sure uniform
convergence. Finally, we show that in a certain sense, Stochastic Gradient
Descent methods minimising $\mathcal{E}$ and $\mathcal{E}_p$ converge towards
(Clarke) critical points of these energies.

本文研究了 Sliced Wasserstein 距离的性质、优化问题和蒙特卡洛逼近，展示了 Stochastic Gradient Descent 方法最小化该距离的收敛性。

离散切片 Wasserstein 损失函数的性质

Properties of Discrete Sliced Wasserstein Losses

Good data augmentation is one of the key factors that lead to the empirical
success of self-supervised representation learning such as contrastive learning
and masked language modeling, yet theoretical understanding of its role in
learning good representations remains limited. Recent work has built the
connection between self-supervised learning and approximating the top
eigenspace of a graph Laplacian operator. Learning a linear probe on top of
such features can naturally be connected to RKHS regression. In this work, we
use this insight to perform a statistical analysis of augmentation-based
pretraining. We start from the isometry property, a key geometric
characterization of the target function given by the augmentation. Our first
main theorem provides, for an arbitrary encoder, near tight bounds for both the
estimation error incurred by fitting the linear probe on top of the encoder,
and the approximation error entailed by the fitness of the RKHS the encoder
learns. Our second main theorem specifically addresses the case where the
encoder extracts the top-d eigenspace of a Monte-Carlo approximation of the
underlying kernel with the finite pretraining samples. Our analysis completely
disentangles the effects of the model and the augmentation. A key ingredient in
our analysis is the augmentation complexity, which we use to quantitatively
compare different augmentations and analyze their impact on downstream
performance on synthetic and real datasets.

本文从 data augmentation 在 self-supervised learning 中的作用入手，结合 RKHS regression 和 Monte-Carlo approximation ，分析了不同 data augmentation 对模型性能的影响。