Fine-tuning from a collection of models pre-trained on different domains (a
"model zoo") is emerging as a technique to improve test accuracy in the
low-data regime. However, model selection, i.e. how to pre-select the right
model to fine-tune from a model zoo without performing any training, remains an
open topic. We use a linearized framework to approximate fine-tuning, and
introduce two new baselines for model selection -- Label-Gradient and
Label-Feature Correlation. Since all model selection algorithms in the
literature have been tested on different use-cases and never compared directly,
we introduce a new comprehensive benchmark for model selection comprising of:
i) A model zoo of single and multi-domain models, and ii) Many target tasks.
Our benchmark highlights accuracy gain with model zoo compared to fine-tuning
Imagenet models. We show our model selection baseline can select optimal models
to fine-tune in few selections and has the highest ranking correlation to
fine-tuning accuracy compared to existing algorithms.

研究通过从不同领域预训练的模型集合（模型库）微调模型是提高低数据量情况下的测试准确性的一种技术，但如何在不进行任何训练的情况下从模型库中预选正确的模型仍然是一个开放的课题。本文使用线性化框架来近似微调，并介绍了两个新的模型选择基准 ——Label-Gradient 和 Label-Feature Correlation，并提出了一个包含单一和多领域模型库以及许多目标任务的新综合基准。该基准突出了与 fine-tuning Imagenet 模型相比使用模型库的准确率提升。我们展示了我们的模型选择基准可以通过少数的选择选择最佳的微调模型，并与现有算法相比具有最高的 fine-tuning 准确度相关性排名。