Conventional dataset distillation requires significant computational
resources and assumes access to the entire dataset, an assumption impractical
as it presumes all data resides on a central server. In this paper, we focus on
dataset distillation in practical scenarios with access to only a fraction of
the entire dataset. We introduce a novel distillation method that augments the
conventional process by incorporating general model knowledge via the addition
of Deep KKT (DKKT) loss. In practical settings, our approach showed improved
performance compared to the baseline distribution matching distillation method
on the CIFAR-10 dataset. Additionally, we present experimental evidence that
Deep Support Vectors (DSVs) offer unique information to the original
distillation, and their integration results in enhanced performance.

在实际场景中仅能访问整个数据集的一部分的情况下，我们提出了一种新的数据集提炼方法，通过添加深度 KKT（DKKT）损失来增强传统过程中的一般模型知识。在 CIFAR-10 数据集上的实验表明，我们的方法在实际设置下相较于基线分布匹配提炼方法表现出更好的性能，并且我们还呈现了深度支持向量（DSVs）在原始提炼中提供了独特信息的实验证据，它们的整合结果提升了性能。