Dataset distillation reduces the storage and computational consumption of
training a network by generating a small surrogate dataset that encapsulates
rich information of the original large-scale one. However, previous
distillation methods heavily rely on the sample-wise iterative optimization
scheme. As the images-per-class (IPC) setting or image resolution grows larger,
the necessary computation will demand overwhelming time and resources. In this
work, we intend to incorporate generative diffusion techniques for computing
the surrogate dataset. Observing that key factors for constructing an effective
surrogate dataset are representativeness and diversity, we design additional
minimax criteria in the generative training to enhance these facets for the
generated images of diffusion models. We present a theoretical model of the
process as hierarchical diffusion control demonstrating the flexibility of the
diffusion process to target these criteria without jeopardizing the
faithfulness of the sample to the desired distribution. The proposed method
achieves state-of-the-art validation performance while demanding much less
computational resources. Under the 100-IPC setting on ImageWoof, our method
requires less than one-twentieth the distillation time of previous methods, yet
yields even better performance. Source code available in
this https URL

通过使用生成扩散技术计算替代数据集，本研究提出了一种新的数据集提炼方法，以增强生成图像的代表性和多样性，并在要求更少计算资源的同时，实现了最新的验证性能。