During the continuous evolution of one organism's ancestry, its genes
accumulate extensive experiences and knowledge, enabling newborn descendants to
rapidly adapt to their specific environments. Motivated by this observation, we
propose a novel machine learning paradigm \textit{Learngene} to enable learning
models to incorporate three key characteristics of genes. (i) Accumulating: the
knowledge is accumulated during the continuous learning of an \textbf{ancestry
model}. (ii) Condensing: the exhaustive accumulated knowledge is condensed into
a much more compact information piece, \ie \textbf{learngene}. (iii):
Inheriting: the condensed \textbf{learngene} is inherited to make it easier for
\textbf{descendant models} to adapt to new environments. Since accumulating has
been studied in some well-developed paradigms like large-scale pre-training and
lifelong learning, we focus on condensing and inheriting, which induces three
key issues and we provide the preliminary solutions to these issues in this
paper: (i) \textit{Learngene} Form: the \textbf{learngene} is set to a few
integral layers that can preserve the most commonality. (ii) \textit{Learngene}
Condensing: we identify which layers among the ancestry model have the most
similarity as one pseudo descendant model. (iii) \textit{Learngene} Inheriting:
to construct distinct descendant models for specific downstream tasks, we stack
some randomly initialized layers to the \textbf{learngene} layers. Extensive
experiments of various settings, including using different network
architectures like Vision Transformer (ViT) and Convolutional Neural Networks
(CNNs) on different datasets, are carried out to confirm five advantages and
two characteristics of \textit{Learngene}.

提出了一种新的机器学习范式 Learngene，其主要特点是能够在一个祖先模型的连续学习期间积累知识，把这些经验压缩成紧凑的信息元素（即 Learngene），并将其遗传给后代模型，使其更容易适应新环境。