Conventional dataset distillation requires significant computational resources and assumes access to the entire dataset, an assumption impractical as it presumes all data resides on a central server. In this paper, we focus on dataset distillation in practical scenarios with access to only a fraction of the entire dataset. We introduce a novel distillation method that augments the conventional process by incorporating general model knowledge via the addition of Deep KKT (DKKT) loss. In practical settings, our approach showed improved performance compared to the baseline distribution matching distillation method on the CIFAR-10 dataset. Additionally, we present experimental evidence that Deep Support Vectors (DSVs) offer unique information to the original distillation, and their integration results in enhanced performance.

在实际场景中仅能访问整个数据集的一部分的情况下，我们提出了一种新的数据集提炼方法，通过添加深度KKT（DKKT）损失来增强传统过程中的一般模型知识。在CIFAR-10数据集上的实验表明，我们的方法在实际设置下相较于基线分布匹配提炼方法表现出更好的性能，并且我们还呈现了深度支持向量（DSVs）在原始提炼中提供了独特信息的实验证据，它们的整合结果提升了性能。

基于深度支持向量的实用数据集蒸馏