To provide a foundation for the research of deep learning models, the
construction of model pool is an essential step. This paper proposes a
Training-Free and Efficient Model Generation and Enhancement Scheme (MGE). This
scheme primarily considers two aspects during the model generation process: the
distribution of model parameters and model performance. Experiments result
shows that generated models are comparable to models obtained through normal
training, and even superior in some cases. Moreover, the time consumed in
generating models accounts for only 1\% of the time required for normal model
training. More importantly, with the enhancement of Evolution-MGE, generated
models exhibits competitive generalization ability in few-shot tasks. And the
behavioral dissimilarity of generated models has the potential of adversarial
defense.

这篇论文提出了一种无需训练且高效的模型生成和增强方案（MGE），通过考虑模型参数分布和性能，在模型生成过程中生成的模型在某些情况下不仅与正常训练得到的模型相媲美，甚至更优。与正常模型训练所需时间相比，生成模型仅需 1% 的时间。在 Few-shot 任务中，使用 Evolution-MGE 进行增强后的生成模型展现出有竞争力的泛化能力，同时生成模型之间的行为差异有潜在的对抗防御能力。

MGE: 一种无需训练的高效模型生成和增强方案

MGE: A Training-Free and Efficient Model Generation and Enhancement  Scheme

Dataset distillation, a pragmatic approach in machine learning, aims to
create a smaller synthetic dataset from a larger existing dataset. However,
existing distillation methods primarily adopt a model-based paradigm, where the
synthetic dataset inherits model-specific biases, limiting its generalizability
to alternative models. In response to this constraint, we propose a novel
methodology termed "model pool". This approach involves selecting models from a
diverse model pool based on a specific probability distribution during the data
distillation process. Additionally, we integrate our model pool with the
established knowledge distillation approach and apply knowledge distillation to
the test process of the distilled dataset. Our experimental results validate
the effectiveness of the model pool approach across a range of existing models
while testing, demonstrating superior performance compared to existing
methodologies.

使用模型池方法进行数据蒸馏，结合知识蒸馏技术进行测试，从而创造出小规模合成数据集，在性能上表现优于现有方法。