Neighbor embeddings are a family of methods for visualizing complex
high-dimensional datasets using $k$NN graphs. To find the low-dimensional
embedding, these algorithms combine an attractive force between neighboring
pairs of points with a repulsive force between all points. One of the most
popular examples of such algorithms is t-SNE. Here we empirically show that
changing the balance between the attractive and the repulsive forces in t-SNE
using the exaggeration parameter yields a spectrum of embeddings, which is
characterized by a simple trade-off: stronger attraction can better represent
continuous manifold structures, while stronger repulsion can better represent
discrete cluster structures and yields higher $k$NN recall. We find that UMAP
embeddings correspond to t-SNE with increased attraction; mathematical analysis
shows that this is because the negative sampling optimisation strategy employed
by UMAP strongly lowers the effective repulsion. Likewise, ForceAtlas2,
commonly used for visualizing developmental single-cell transcriptomic data,
yields embeddings corresponding to t-SNE with the attraction increased even
more. At the extreme of this spectrum lie Laplacian Eigenmaps. Our results
demonstrate that many prominent neighbor embedding algorithms can be placed
onto the attraction-repulsion spectrum, and highlight the inherent trade-offs
between them.

本研究介绍了邻居嵌入算法族，针对不同的连续流形结构和离散聚类结构之间的权衡关系，以 $t-SNE$ 为例，通过夸张参数来调节吸引力与排斥力之间的平衡，得到了一系列的嵌入，UMAP 算法通过负采样策略降低有效排斥，较高的吸引力偏重于表达连续流形结构，而较高的排斥力偏重于表达离散聚类结构，邻居嵌入算法族可以被放置在这个平衡的关系上，探讨常见的算法在这个关系上的位置。

邻居嵌入中的吸引 - 排斥谱

Attraction-Repulsion Spectrum in Neighbor Embeddings

Deep clustering has increasingly been demonstrating superiority over
conventional shallow clustering algorithms. Deep clustering algorithms usually
combine representation learning with deep neural networks to achieve this
performance, typically optimizing a clustering and non-clustering loss. In such
cases, an autoencoder is typically connected with a clustering network, and the
final clustering is jointly learned by both the autoencoder and clustering
network. Instead, we propose to learn an autoencoded embedding and then search
this further for the underlying manifold. For simplicity, we then cluster this
with a shallow clustering algorithm, rather than a deeper network. We study a
number of local and global manifold learning methods on both the raw data and
autoencoded embedding, concluding that UMAP in our framework is best able to
find the most clusterable manifold in the embedding, suggesting local manifold
learning on an autoencoded embedding is effective for discovering higher
quality discovering clusters. We quantitatively show across a range of image
and time-series datasets that our method has competitive performance against
the latest deep clustering algorithms, including out-performing current
state-of-the-art on several. We postulate that these results show a promising
research direction for deep clustering. The code can be found at
this https URL

本研究提出了在深度非监督聚类中使用 autoencoded embedding 和 UMAP 的方法，该方法可以优于目前常见的深度聚类算法，并在多个数据集上取得了有竞争力的结果，为深度聚类提供了有前途的研究方向。

N2D：通过对自编码嵌入的局部流形聚类来实现（不太）深的聚类

N2D: (Not Too) Deep Clustering via Clustering the Local Manifold of an  Autoencoded Embedding

UMAP (Uniform Manifold Approximation and Projection) is a novel manifold
learning technique for dimension reduction. UMAP is constructed from a
theoretical framework based in Riemannian geometry and algebraic topology. The
result is a practical scalable algorithm that applies to real world data. The
UMAP algorithm is competitive with t-SNE for visualization quality, and
arguably preserves more of the global structure with superior run time
performance. Furthermore, UMAP has no computational restrictions on embedding
dimension, making it viable as a general purpose dimension reduction technique
for machine learning.

UMAP 是一种基于黎曼几何和代数拓扑的流形学习技术，可以作为机器学习中通用的降维算法，具有与 t-SNE 相当的可视化质量和更好的整体结构保留能力，并且没有关于嵌入维度的计算限制。