The pre-training paradigm fine-tunes the models trained on large-scale
datasets to downstream tasks with enhanced performance. It transfers all
knowledge to downstream tasks without discriminating which part is necessary or
unnecessary, which may lead to negative transfer. In comparison, knowledge
transfer in nature is much more efficient. When passing genetic information to
descendants, ancestors encode only the essential knowledge into genes, which
act as the medium. Inspired by that, we adopt a recent concept called
``learngene'' and refine its structures by mimicking the structures of natural
genes. We propose the Genetic Transfer Learning (GTL) -- a framework to copy
the evolutionary process of organisms into neural networks. GTL trains a
population of networks, selects superior learngenes by tournaments, performs
learngene mutations, and passes the learngenes to next generations. Finally, we
successfully extract the learngenes of VGG11 and ResNet12. We show that the
learngenes bring the descendant networks instincts and strong learning ability:
with 20% parameters, the learngenes bring 12% and 16% improvements of accuracy
on CIFAR-FS and miniImageNet. Besides, the learngenes have the scalability and
adaptability on the downstream structure of networks and datasets. Overall, we
offer a novel insight that transferring core knowledge via learngenes may be
sufficient and efficient for neural networks.

通过模仿自然基因的结构，我们采用名为 “learngene” 的新概念，并提出一种名为遗传传输学习（GTL）的框架，将有机体的进化过程复制到神经网络中。我们成功提取了 VGG11 和 ResNet12 的 learngenes，并证明了通过 learngenes 传递核心知识可能足够高效地提高神经网络的准确性。