Dataset condensation addresses the problem of data burden by learning a small
synthetic training set that preserves essential knowledge from the larger real
training set. To date, the state-of-the-art (SOTA) results are often yielded by
optimization-oriented methods, but their inefficiency hinders their application
to realistic datasets. On the other hand, the Distribution-Matching (DM)
methods show remarkable efficiency but sub-optimal results compared to
optimization-oriented methods. In this paper, we reveal the limitations of
current DM-based methods from the inner-class and inter-class views, i.e.,
Persistent Training and Distribution Shift. To address these problems, we
propose a new DM-based method named Dual-view distribution AligNment for
dataset CondEnsation (DANCE), which exploits a few pre-trained models to
improve DM from both inner-class and inter-class views. Specifically, from the
inner-class view, we construct multiple "middle encoders" to perform pseudo
long-term distribution alignment, making the condensed set a good proxy of the
real one during the whole training process; while from the inter-class view, we
use the expert models to perform distribution calibration, ensuring the
synthetic data remains in the real class region during condensing. Experiments
demonstrate the proposed method achieves a SOTA performance while maintaining
comparable efficiency with the original DM across various scenarios. Source
codes are available at this https URL

从内部和外部视角揭示了当前基于分布匹配的方法在持久化训练和分布偏移方面的局限性，提出了一种新的基于双视图分布对齐的数据集简化方法 (DANCE)，通过利用多个预训练模型来改进分布匹配，从而取得了状态的结果，并在保持与原始分布匹配相当的效率下实现了流行的性能。