The ubiquity of large-scale Pre-Trained Models (PTMs) is on the rise,
sparking interest in model hubs, and dedicated platforms for hosting PTMs.
Despite this trend, a comprehensive exploration of the challenges that users
encounter and how the community leverages PTMs remains lacking. To address this
gap, we conducted an extensive mixed-methods empirical study by focusing on
discussion forums and the model hub of HuggingFace, the largest public model
hub. Based on our qualitative analysis, we present a taxonomy of the challenges
and benefits associated with PTM reuse within this community. We then conduct a
quantitative study to track model-type trends and model documentation evolution
over time. Our findings highlight prevalent challenges such as limited guidance
for beginner users, struggles with model output comprehensibility in training
or inference, and a lack of model understanding. We also identified interesting
trends among models where some models maintain high upload rates despite a
decline in topics related to them. Additionally, we found that despite the
introduction of model documentation tools, its quantity has not increased over
time, leading to difficulties in model comprehension and selection among users.
Our study sheds light on new challenges in reusing PTMs that were not reported
before and we provide recommendations for various stakeholders involved in PTM
reuse.

模型复用方面，我们通过一项综合的混合方法实证研究，通过关注讨论论坛和 HuggingFace 模型中心，提出了一种挑战和益处的分类法，并进行了定量研究以追踪模型类型趋势和模型文档的演变。

HuggingFace 社区中深度学习模型重用的挑战、利益与趋势

Deep Learning Model Reuse in the HuggingFace Community: Challenges,  Benefit and Trends

Model hubs with many pre-trained models (PTMs) have become a cornerstone of
deep learning. Although built at a high cost, they remain
\emph{under-exploited} -- practitioners usually pick one PTM from the provided
model hub by popularity and then fine-tune the PTM to solve the target task.
This na\"ive but common practice poses two obstacles to full exploitation of
pre-trained model hubs: first, the PTM selection by popularity has no
optimality guarantee, and second, only one PTM is used while the remaining PTMs
are ignored. An alternative might be to consider all possible combinations of
PTMs and extensively fine-tune each combination, but this would not only be
prohibitive computationally but may also lead to statistical over-fitting. In
this paper, we propose a new paradigm for exploiting model hubs that is
intermediate between these extremes. The paradigm is characterized by two
aspects: (1) We use an evidence maximization procedure to estimate the maximum
value of label evidence given features extracted by pre-trained models. This
procedure can rank all the PTMs in a model hub for various types of PTMs and
tasks \emph{before fine-tuning}. (2) The best ranked PTM can either be
fine-tuned and deployed if we have no preference for the model's architecture
or the target PTM can be tuned by the top $K$ ranked PTMs via a Bayesian
procedure that we propose. This procedure, which we refer to as
\emph{B-Tuning}, not only improves upon specialized methods designed for tuning
homogeneous PTMs, but also applies to the challenging problem of tuning
heterogeneous PTMs where it yields a new level of benchmark performance.

本文提出了一种新的模型中心的利用范式，该范式既考虑了模型的性能，又避免了计算负担和统计过拟合问题。该方法使用证据最大化程序对预训练模型提取的特征进行评估并排名所有预训练模型，然后利用贝叶斯程序进行调优。该过程不仅适用于同构预训练模型的调优，而且还适用于异构预训练模型的调优。