Pre-trained deep learning (DL) models are increasingly accessible in public
repositories, i.e., model zoos. Given a new prediction task, finding the best
model to fine-tune can be computationally intensive and costly, especially when
the number of pre-trained models is large. Selecting the right pre-trained
models is crucial, yet complicated by the diversity of models from various
model families (like ResNet, Vit, Swin) and the hidden relationships between
models and datasets. Existing methods, which utilize basic information from
models and datasets to compute scores indicating model performance on target
datasets, overlook the intrinsic relationships, limiting their effectiveness in
model selection. In this study, we introduce TransferGraph, a novel framework
that reformulates model selection as a graph learning problem. TransferGraph
constructs a graph using extensive metadata extracted from models and datasets,
while capturing their inherent relationships. Through comprehensive experiments
across 16 real datasets, both images and texts, we demonstrate TransferGraph's
effectiveness in capturing essential model-dataset relationships, yielding up
to a 32% improvement in correlation between predicted performance and the
actual fine-tuning results compared to the state-of-the-art methods.

通过分析预先训练的深度学习模型的元数据，并捕捉模型和数据集之间的内在关系，研究人员提出了一种名为 TransferGraph 的新框架，将模型选择问题重新定义为图学习问题，并在 16 个真实数据集上进行了广泛实验，发现相较于现有方法，TransferGraph 在预测性能和实际微调结果之间的相关性上有 32% 的提升。