The rapid growth of neural network models shared on the internet has made
model weights an important data modality. However, this information is
underutilized as the weights are uninterpretable, and publicly available models
are disorganized. Inspired by Darwin's tree of life, we define the Model Tree
which describes the origin of models i.e., the parent model that was used to
fine-tune the target model. Similarly to the natural world, the tree structure
is unknown. In this paper, we introduce the task of Model Tree Heritage
Recovery (MoTHer Recovery) for discovering Model Trees in the ever-growing
universe of neural networks. Our hypothesis is that model weights encode this
information, the challenge is to decode the underlying tree structure given the
weights. Beyond the immediate application of model authorship attribution,
MoTHer recovery holds exciting long-term applications akin to indexing the
internet by search engines. Practically, for each pair of models, this task
requires: i) determining if they are related, and ii) establishing the
direction of the relationship. We find that certain distributional properties
of the weights evolve monotonically during training, which enables us to
classify the relationship between two given models. MoTHer recovery
reconstructs entire model hierarchies, represented by a directed tree, where a
parent model gives rise to multiple child models through additional training.
Our approach successfully reconstructs complex Model Trees, as well as the
structure of "in-the-wild" model families such as Llama 2 and Stable Diffusion.

神经网络模型的快速增长使模型权重成为重要的数据形式。我们通过定义模型树描述模型的起源来解决网络上公开的模型数据无法解释和组织的问题。我们介绍了模型树继承恢复任务，旨在根据模型权重解码模型的树结构，从而有助于模型的作者鉴定和互联网索引等应用。通过对权重的分布特性进行分类，我们能够确定两个模型之间的关系和关系的方向。MoTHer 恢复将模型层次结构表示为一个有向树，成功地重建了复杂的模型树和 “野外” 模型族群的结构，如 Llama 2 和稳定扩散。