Herein, we propose a novel dataset distillation method for constructing small
informative datasets that preserve the information of the large original
datasets. The development of deep learning models is enabled by the
availability of large-scale datasets. Despite unprecedented success,
large-scale datasets considerably increase the storage and transmission costs,
resulting in a cumbersome model training process. Moreover, using raw data for
training raises privacy and copyright concerns. To address these issues, a new
task named dataset distillation has been introduced, aiming to synthesize a
compact dataset that retains the essential information from the large original
dataset. State-of-the-art (SOTA) dataset distillation methods have been
proposed by matching gradients or network parameters obtained during training
on real and synthetic datasets. The contribution of different network
parameters to the distillation process varies, and uniformly treating them
leads to degraded distillation performance. Based on this observation, we
propose an importance-aware adaptive dataset distillation (IADD) method that
can improve distillation performance by automatically assigning importance
weights to different network parameters during distillation, thereby
synthesizing more robust distilled datasets. IADD demonstrates superior
performance over other SOTA dataset distillation methods based on parameter
matching on multiple benchmark datasets and outperforms them in terms of
cross-architecture generalization. In addition, the analysis of self-adaptive
weights demonstrates the effectiveness of IADD. Furthermore, the effectiveness
of IADD is validated in a real-world medical application such as COVID-19
detection.

我们提出了一种新颖的数据集精炼方法，可以构建小型信息丰富的数据集，以保留大型原始数据集的信息。这种方法可以改善数据集精炼的性能，通过自动分配不同网络参数的重要性权重来合成更加稳健的精炼数据集，在多个基准数据集上表现优于其他最先进的数据集精炼方法，并在跨架构泛化方面胜过它们。此外，我们实验证明了自适应权重的有效性，并在 COVID-19 检测等现实世界医学应用中验证了 IADD 的有效性。