The rapid evolution of deep learning and large language models has led to an exponential growth in the demand for training data, prompting the development of Dataset Distillation methods to address the challenges of managing large datasets. Among these, Matching Training Trajectories (MTT) has been a prominent approach, which replicates the training trajectory of an expert network on real data with a synthetic dataset. However, our investigation found that this method suffers from three significant limitations: 1. Instability of expert trajectory generated by Stochastic Gradient Descent (SGD); 2. Low convergence speed of the distillation process; 3. High storage consumption of the expert trajectory. To address these issues, we offer a new perspective on understanding the essence of Dataset Distillation and MTT through a simple transformation of the objective function, and introduce a novel method called Matching Convexified Trajectory (MCT), which aims to provide better guidance for the student trajectory. MCT leverages insights from the linearized dynamics of Neural Tangent Kernel methods to create a convex combination of expert trajectories, guiding the student network to converge rapidly and stably. This trajectory is not only easier to store, but also enables a continuous sampling strategy during distillation, ensuring thorough learning and fitting of the entire expert trajectory. Comprehensive experiments across three public datasets validate the superiority of MCT over traditional MTT methods.

通过对目标函数进行简单的转换和利用 MCT （Matching Convexified Trajectory）方法，研究论文发现 MTT 方法存在的三个主要问题：Stochastic Gradient Descent（SGD）生成的专家轨迹不稳定，提取过程的收敛速度慢以及专家轨迹的存储消耗高，并提出了一种能够快速且稳定地引导学生网络收敛的方法。实验结果显示，MCT 方法在三个公共数据集上相较于传统的 MTT 方法具有优越性。

稳定且存储有效的数据集蒸馏:匹配凸化轨迹