Background: Collaborative Software Package Registries (SPRs) are an integral
part of the software supply chain. Much engineering work synthesizes SPR
package into applications. Prior research has examined SPRs for traditional
software, such as NPM (JavaScript) and PyPI (Python). Pre-Trained Model (PTM)
Registries are an emerging class of SPR of increasing importance, because they
support the deep learning supply chain.
Aims: Recent empirical research has examined PTM registries in ways such as
vulnerabilities, reuse processes, and evolution. However, no existing research
synthesizes them to provide a systematic understanding of the current
knowledge. Some of the existing research includes qualitative claims lacking
quantitative analysis. Our research fills these gaps by providing a knowledge
synthesis and quantitative analyses.
Methods: We first conduct a systematic literature review (SLR). We then
observe that some of the claims are qualitative. We identify quantifiable
metrics associated with those claims, and measure in order to substantiate
these claims.
Results: From our SLR, we identify 12 claims about PTM reuse on the
HuggingFace platform, 4 of which lack quantitative validation. We successfully
test 3 of these claims through a quantitative analysis, and directly compare
one with traditional software. Our findings corroborate qualitative claims with
quantitative measurements. Our findings are: (1) PTMs have a much higher
turnover rate than traditional software, indicating a dynamic and rapidly
evolving reuse environment within the PTM ecosystem; and (2) There is a strong
correlation between documentation quality and PTM popularity.
Conclusions: We confirm qualitative research claims with concrete metrics,
supporting prior qualitative and case study research. Our measures show further
dynamics of PTM reuse, inspiring research infrastructure and new measures.

我们通过系统文献综述和定量分析验证了关于 Pre-Trained Model （PTM）的重复使用的定性研究结论，证实 PTM 具有比传统软件更高的更新速度，并且文档质量与 PTM 的受欢迎程度存在强相关性。

Hugging Face 的相关知识：系统文献综述与定性声明的定量验证

What do we know about Hugging Face? A systematic literature review and  quantitative validation of qualitative claims

The ubiquity of large-scale Pre-Trained Models (PTMs) is on the rise,
sparking interest in model hubs, and dedicated platforms for hosting PTMs.
Despite this trend, a comprehensive exploration of the challenges that users
encounter and how the community leverages PTMs remains lacking. To address this
gap, we conducted an extensive mixed-methods empirical study by focusing on
discussion forums and the model hub of HuggingFace, the largest public model
hub. Based on our qualitative analysis, we present a taxonomy of the challenges
and benefits associated with PTM reuse within this community. We then conduct a
quantitative study to track model-type trends and model documentation evolution
over time. Our findings highlight prevalent challenges such as limited guidance
for beginner users, struggles with model output comprehensibility in training
or inference, and a lack of model understanding. We also identified interesting
trends among models where some models maintain high upload rates despite a
decline in topics related to them. Additionally, we found that despite the
introduction of model documentation tools, its quantity has not increased over
time, leading to difficulties in model comprehension and selection among users.
Our study sheds light on new challenges in reusing PTMs that were not reported
before and we provide recommendations for various stakeholders involved in PTM
reuse.

模型复用方面，我们通过一项综合的混合方法实证研究，通过关注讨论论坛和 HuggingFace 模型中心，提出了一种挑战和益处的分类法，并进行了定量研究以追踪模型类型趋势和模型文档的演变。

HuggingFace 社区中深度学习模型重用的挑战、利益与趋势

Deep Learning Model Reuse in the HuggingFace Community: Challenges,  Benefit and Trends

Deep Neural Networks (DNNs) are being adopted as components in software
systems. Creating and specializing DNNs from scratch has grown increasingly
difficult as state-of-the-art architectures grow more complex. Following the
path of traditional software engineering, machine learning engineers have begun
to reuse large-scale pre-trained models (PTMs) and fine-tune these models for
downstream tasks. Prior works have studied reuse practices for traditional
software packages to guide software engineers towards better package
maintenance and dependency management. We lack a similar foundation of
knowledge to guide behaviors in pre-trained model ecosystems.
In this work, we present the first empirical investigation of PTM reuse. We
interviewed 12 practitioners from the most popular PTM ecosystem, Hugging Face,
to learn the practices and challenges of PTM reuse. From this data, we model
the decision-making process for PTM reuse. Based on the identified practices,
we describe useful attributes for model reuse, including provenance,
reproducibility, and portability. Three challenges for PTM reuse are missing
attributes, discrepancies between claimed and actual performance, and model
risks. We substantiate these identified challenges with systematic measurements
in the Hugging Face ecosystem. Our work informs future directions on optimizing
deep learning ecosystems by automated measuring useful attributes and potential
attacks, and envision future research on infrastructure and standardization for
model registries.

本文是关于使用预训练模型的实践和挑战，通过对 Hugging Face 生态系统中从业者的访谈和系统化的测量，确定了模型重用的实践和决策制定过程，并提出了模型重用的有用属性和挑战，以及对深度学习生态系统自动化地测量有用属性和潜在攻击的未来方向进行了展望。