Dataset distillation has emerged as a promising approach in deep learning,
enabling efficient training with small synthetic datasets derived from larger
real ones. Particularly, distribution matching-based distillation methods
attract attention thanks to its effectiveness and low computational cost.
However, these methods face two primary limitations: the dispersed feature
distribution within the same class in synthetic datasets, reducing class
discrimination, and an exclusive focus on mean feature consistency, lacking
precision and comprehensiveness. To address these challenges, we introduce two
novel constraints: a class centralization constraint and a covariance matching
constraint. The class centralization constraint aims to enhance class
discrimination by more closely clustering samples within classes. The
covariance matching constraint seeks to achieve more accurate feature
distribution matching between real and synthetic datasets through local feature
covariance matrices, particularly beneficial when sample sizes are much smaller
than the number of features. Experiments demonstrate notable improvements with
these constraints, yielding performance boosts of up to 6.6% on CIFAR10, 2.9%
on SVHN, 2.5% on CIFAR100, and 2.5% on TinyImageNet, compared to the
state-of-the-art relevant methods. In addition, our method maintains robust
performance in cross-architecture settings, with a maximum performance drop of
1.7% on four architectures. Code is available at
this https URL

通过使用类居中约束和协方差匹配约束，提出的数据集精馏方法在解决分散特征分布和精确特征分布匹配两个主要限制方面表现出色，与相关方法相比，在 CIFAR10、SVHN、CIFAR100 和 TinyImageNet 上的性能提升分别达到最大为 6.6%、2.9%、2.5% 和 2.5%，并且在四个体系结构上保持稳定表现，最大性能降低为 1.7%。