In this paper, we study a novel task that enables partial knowledge transfer
from pre-trained models, which we term as Partial Network Cloning (PNC). Unlike
prior methods that update all or at least part of the parameters in the target
network throughout the knowledge transfer process, PNC conducts partial
parametric "cloning" from a source network and then injects the cloned module
to the target, without modifying its parameters. Thanks to the transferred
module, the target network is expected to gain additional functionality, such
as inference on new classes; whenever needed, the cloned module can be readily
removed from the target, with its original parameters and competence kept
intact. Specifically, we introduce an innovative learning scheme that allows us
to identify simultaneously the component to be cloned from the source and the
position to be inserted within the target network, so as to ensure the optimal
performance. Experimental results on several datasets demonstrate that, our
method yields a significant improvement of 5% in accuracy and 50% in locality
when compared with parameter-tuning based methods. Our code is available at
this https URL

本文研究了一种新颖的任务，即部分知识转移，称为 “Partial Network Cloning (PNC)”。我们介绍了一种创新的学习方案，允许我们同时识别要从源代码中克隆的组件和要插入到目标网络中的位置，以确保最佳性能。在多个数据集上的实验结果表明，与基于参数调整的方法相比，我们的方法在准确度和局部性能方面都有显著提高。