Recent advances in self-supervised learning have highlighted the efficacy of
data augmentation in learning data representation from unlabeled data. Training
a linear model atop these enhanced representations can yield an adept
classifier. Despite the remarkable empirical performance, the underlying
mechanisms that enable data augmentation to unravel nonlinear data structures
into linearly separable representations remain elusive. This paper seeks to
bridge this gap by investigating under what conditions learned representations
can linearly separate manifolds when data is drawn from a multi-manifold model.
Our investigation reveals that data augmentation offers additional information
beyond observed data and can thus improve the information-theoretic optimal
rate of linear separation capacity. In particular, we show that self-supervised
learning can linearly separate manifolds with a smaller distance than
unsupervised learning, underscoring the additional benefits of data
augmentation. Our theoretical analysis further underscores that the performance
of downstream linear classifiers primarily hinges on the linear separability of
data representations rather than the size of the labeled data set, reaffirming
the viability of constructing efficient classifiers with limited labeled data
amid an expansive unlabeled data set.

研究表明，自监督学习和数据增强在从无标签数据中学习数据表示方面具有高效性，通过在这些增强表示之上训练线性模型可以得到熟练的分类器。本文探究了数据增强如何在多流形模型中实现线性分离，发现数据增强提供了超越观察数据的额外信息，可以改善线性分离能力。自监督学习可以通过更小的距离线性分离流形，并强调数据增强的附加好处。理论分析进一步证明下游线性分类器的性能主要取决于数据表示的线性可分性，而不仅仅是标记数据集的大小，验证了在广阔的无标签数据集中利用有限标记数据构建高效分类器的可行性。

自监督表征学习的线性分隔能力

On Linear Separation Capacity of Self-Supervised Representation Learning

Although data diffusion embeddings are ubiquitous in unsupervised learning
and have proven to be a viable technique for uncovering the underlying
intrinsic geometry of data, diffusion embeddings are inherently limited due to
their discrete nature. To this end, we propose neural FIM, a method for
computing the Fisher information metric (FIM) from point cloud data - allowing
for a continuous manifold model for the data. Neural FIM creates an extensible
metric space from discrete point cloud data such that information from the
metric can inform us of manifold characteristics such as volume and geodesics.
We demonstrate Neural FIM's utility in selecting parameters for the PHATE
visualization method as well as its ability to obtain information pertaining to
local volume illuminating branching points and cluster centers embeddings of a
toy dataset and two single-cell datasets of IPSC reprogramming and PBMCs
(immune cells).

提出一种名为神经 FIM 的新方法，可以从离散点云数据中计算 Fisher 信息度量，并创建一个可扩展的度量空间，以建立连续流形模型来生成嵌入数据，以量化该流形的各种特征，证明了神经 FIM 在选择 PHATE 可视化方法的参数方面的实用性以及在 IPSC 重编程和 PBMCs（免疫细胞）等单细胞数据集的分支点和聚类中心嵌入的局部体积方面获取的信息。

神经 FIM：从点云数据中学习 Fisher 信息度量

Neural FIM for learning Fisher Information Metrics from point cloud data

3D point cloud - a new signal representation of volumetric objects - is a
discrete collection of triples marking exterior object surface locations in 3D
space. Conventional imperfect acquisition processes of 3D point cloud - e.g.,
stereo-matching from multiple viewpoint images or depth data acquired directly
from active light sensors - imply non-negligible noise in the data. In this
paper, we adopt a previously proposed low-dimensional manifold model for the
surface patches in the point cloud and seek self-similar patches to denoise
them simultaneously using the patch manifold prior. Due to discrete
observations of the patches on the manifold, we approximate the manifold
dimension computation defined in the continuous domain with a patch-based graph
Laplacian regularizer and propose a new discrete patch distance measure to
quantify the similarity between two same-sized surface patches for graph
construction that is robust to noise. We show that our graph Laplacian
regularizer has a natural graph spectral interpretation, and has desirable
numerical stability properties via eigenanalysis. Extensive simulation results
show that our proposed denoising scheme can outperform state-of-the-art methods
in objective metrics and can better preserve visually salient structural
features like edges.

本文提出一种基于低维流形模型和图拉普拉斯正则化器的 3D 点云去噪方法，通过近似离散观测的流形维度计算，并且采用一种新的离散补丁距离度量来构建一个抗噪声的图形结构，并取得了比当前最先进方法更好的性能和更好的结构特征保持。