The rapid growth of neural network models shared on the internet has made
model weights an important data modality. However, this information is
underutilized as the weights are uninterpretable, and publicly available models
are disorganized. Inspired by Darwin's tree of life, we define the Model Tree
which describes the origin of models i.e., the parent model that was used to
fine-tune the target model. Similarly to the natural world, the tree structure
is unknown. In this paper, we introduce the task of Model Tree Heritage
Recovery (MoTHer Recovery) for discovering Model Trees in the ever-growing
universe of neural networks. Our hypothesis is that model weights encode this
information, the challenge is to decode the underlying tree structure given the
weights. Beyond the immediate application of model authorship attribution,
MoTHer recovery holds exciting long-term applications akin to indexing the
internet by search engines. Practically, for each pair of models, this task
requires: i) determining if they are related, and ii) establishing the
direction of the relationship. We find that certain distributional properties
of the weights evolve monotonically during training, which enables us to
classify the relationship between two given models. MoTHer recovery
reconstructs entire model hierarchies, represented by a directed tree, where a
parent model gives rise to multiple child models through additional training.
Our approach successfully reconstructs complex Model Trees, as well as the
structure of "in-the-wild" model families such as Llama 2 and Stable Diffusion.

神经网络模型的快速增长使模型权重成为重要的数据形式。我们通过定义模型树描述模型的起源来解决网络上公开的模型数据无法解释和组织的问题。我们介绍了模型树继承恢复任务，旨在根据模型权重解码模型的树结构，从而有助于模型的作者鉴定和互联网索引等应用。通过对权重的分布特性进行分类，我们能够确定两个模型之间的关系和关系的方向。MoTHer 恢复将模型层次结构表示为一个有向树，成功地重建了复杂的模型树和 “野外” 模型族群的结构，如 Llama 2 和稳定扩散。

关于羊驼起源的模型树遗传恢复

On the Origin of Llamas: Model Tree Heritage Recovery

This paper addresses the trade-off between Accuracy and Transparency for deep
learning applied to sports analytics. Neural nets achieve great predictive
accuracy through deep learning, and are popular in sports analytics. But it is
hard to interpret a neural net model and harder still to extract actionable
insights from the knowledge implicit in it. Therefore, we built a simple and
transparent model that mimics the output of the original deep learning model
and represents the learned knowledge in an explicit interpretable way. Our
mimic model is a linear model tree, which combines a collection of linear
models with a regression-tree structure. The tree version of a neural network
achieves high fidelity, explains itself, and produces insights for expert
stakeholders such as athletes and coaches. We propose and compare several
scalable model tree learning heuristics to address the computational challenge
from datasets with millions of data points.

研究关注于解决深度学习在运动分析中精度和透明度的权衡问题，并且构建一种基于模型树的简单透明模型，以实现高保真输出、自我解释和为运动员和教练等专家利益相关者提供见解的目标。

破解黑匣子：梳理深入体育分析

Cracking the Black Box: Distilling Deep Sports Analytics

Background: Adaptation technique is a crucial task for analogy based
estimation. Current adaptation techniques often use linear size or linear
similarity adjustment mechanisms which are often not suitable for datasets that
have complex structure with many categorical attributes. Furthermore, the use
of nonlinear adaptation technique such as neural network and genetic algorithms
needs many user interactions and parameters optimization for configuring them
(such as network model, number of neurons, activation functions, training
functions, mutation, selection, crossover, ... etc.). Aims: In response to the
abovementioned challenges, the present paper proposes a new adaptation strategy
using Model Tree based attribute distance to adjust estimation by analogy and
derive new estimates. Using Model Tree has an advantage to deal with
categorical attributes, minimize user interaction and improve efficiency of
model learning through classification. Method: Seven well known datasets have
been used with 3-Fold cross validation to empirically validate the proposed
approach. The proposed method has been investigated using various K analogies
from 1 to 3. Results: Experimental results showed that the proposed approach
produced better results when compared with those obtained by using estimation
by analogy based linear size adaptation, linear similarity adaptation,
'regression towards the mean' and null adaptation. Conclusions: Model Tree
could form a useful extension for estimation by analogy especially for complex
data sets with large number of categorical attributes.

本文提出了一种新的模型树属性距离适应策略，用于调整类比估计和生成新的估计，实验结果表明该方法对于具有大量分类属性的复杂数据集尤其有效。