Self-supervised features are the cornerstone of modern machine learning
systems. They are typically pre-trained on data collections whose construction
and curation typically require extensive human effort. This manual process has
some limitations similar to those encountered in supervised learning, e.g., the
crowd-sourced selection of data is costly and time-consuming, preventing
scaling the dataset size. In this work, we consider the problem of automatic
curation of high-quality datasets for self-supervised pre-training. We posit
that such datasets should be large, diverse and balanced, and propose a
clustering-based approach for building ones satisfying all these criteria. Our
method involves successive and hierarchical applications of $k$-means on a
large and diverse data repository to obtain clusters that distribute uniformly
among data concepts, followed by a hierarchical, balanced sampling step from
these clusters. Extensive experiments on three different data domains including
web-based images, satellite images and text show that features trained on our
automatically curated datasets outperform those trained on uncurated data while
being on par or better than ones trained on manually curated data.

自我监督特征是现代机器学习系统的基石，本文提出了一种基于聚类的自动化数据精选方法，通过在大规模和多样化的数据库上应用层次聚类和均衡采样的步骤，构建了大型、多样化且均衡的数据集，实验证明基于这种自动化精选的数据集训练得到的特征在三个不同领域的数据表现优于未精选数据的训练特征，且与手工精选数据的特征相当甚至更好。

自监督学习的自动数据整理：一种基于聚类的方法

Automatic Data Curation for Self-Supervised Learning: A Clustering-Based  Approach

Dimensionality reduction methods are employed to decrease data
dimensionality, either to enhance machine learning performance or to facilitate
data visualization in two or three-dimensional spaces. These methods typically
fall into two categories: feature selection and feature transformation. Feature
selection retains significant features, while feature transformation projects
data into a lower-dimensional space, with linear and nonlinear methods. While
nonlinear methods excel in preserving local structures and capturing nonlinear
relationships, they may struggle with interpreting global structures and can be
computationally intensive. Recent algorithms, such as the t-SNE, UMAP, TriMap,
and PaCMAP prioritize preserving local structures, often at the expense of
accurately representing global structures, leading to clusters being spread out
more in lower-dimensional spaces. Moreover, these methods heavily rely on
hyperparameters, making their results sensitive to parameter settings. To
address these limitations, this study introduces a clustering-based approach,
namely CBMAP (Clustering-Based Manifold Approximation and Projection), for
dimensionality reduction. CBMAP aims to preserve both global and local
structures, ensuring that clusters in lower-dimensional spaces closely resemble
those in high-dimensional spaces. Experimental evaluations on benchmark
datasets demonstrate CBMAP's efficacy, offering speed, scalability, and minimal
reliance on hyperparameters. Importantly, CBMAP enables low-dimensional
projection of test data, addressing a critical need in machine learning
applications. CBMAP is made freely available at
this https URL and can be installed from the Python Package
Directory (PyPI) software repository with the command pip install cbmap.

介绍了一种基于聚类的降维方法 CBMAP，旨在同时保留全局和局部结构，提供速度、可扩展性和对超参数依赖较小的解决方案，满足机器学习应用中对测试数据的低维投影的需求。