Deep learning models often require large amounts of data for training,
leading to increased costs. It is particularly challenging in medical imaging,
i.e., gathering distributed data for centralized training, and meanwhile,
obtaining quality labels remains a tedious job. Many methods have been proposed
to address this issue in various training paradigms, e.g., continual learning,
active learning, and federated learning, which indeed demonstrate certain forms
of the data valuation process. However, existing methods are either overly
intuitive or limited to common clean/toy datasets in the experiments. In this
work, we present two data valuation metrics based on Synaptic Intelligence and
gradient norms, respectively, to study the redundancy in real-world image data.
Novel online and offline data selection algorithms are then proposed via
clustering and grouping based on the examined data values. Our online approach
effectively evaluates data utilizing layerwise model parameter updates and
gradients in each epoch and can accelerate model training with fewer epochs and
a subset (e.g., 19%-59%) of data while maintaining equivalent levels of
accuracy in a variety of datasets. It also extends to the offline coreset
construction, producing subsets of only 18%-30% of the original. The codes for
the proposed adaptive data selection and coreset computation are available
(this https URL).

提出了两种基于神经突触智能和梯度范数的数据估值指标，通过在线聚类和基于考察数据值的分组提出了新的数据选择算法。在线方法使用逐层模型参数更新和每个时期梯度高效地评估数据，可以使用更少的时期和数据子集（例如，19％-59％）加速模型训练，同时在各种数据集中保持等价的水平，还可以扩展到离线 Coreset 构建，只生成原始数据的 18％-30％子集。