Contemporary machine learning requires training large neural networks on
massive datasets and thus faces the challenges of high computational demands.
Dataset distillation, as a recent emerging strategy, aims to compress
real-world datasets for efficient training. However, this line of research
currently struggle with large-scale and high-resolution datasets, hindering its
practicality and feasibility. To this end, we re-examine the existing dataset
distillation methods and identify three properties required for large-scale
real-world applications, namely, realism, diversity, and efficiency. As a
remedy, we propose RDED, a novel computationally-efficient yet effective data
distillation paradigm, to enable both diversity and realism of the distilled
data. Extensive empirical results over various neural architectures and
datasets demonstrate the advancement of RDED: we can distill the full
ImageNet-1K to a small dataset comprising 10 images per class within 7 minutes,
achieving a notable 42% top-1 accuracy with ResNet-18 on a single RTX-4090 GPU
(while the SOTA only achieves 21% but requires 6 hours).

用于大规模真实数据集的数据集压缩方法 RDED 能够在短短 7 分钟内将完整的 ImageNet-1K 压缩为每类只有 10 张图像的小型数据集，并在单个 RTX-4090 GPU 上以 42% 的 Top-1 精度实现了显著的性能提升（而目前最先进方法仅达到 21% 的精度，且需要 6 小时）。