In this paper, we explore a novel knowledge-transfer task, termed as Deep
Model Reassembly (DeRy), for general-purpose model reuse. Given a collection of
heterogeneous models pre-trained from distinct sources and with diverse
architectures, the goal of DeRy, as its name implies, is to first dissect each
model into distinctive building blocks, and then selectively reassemble the
derived blocks to produce customized networks under both the hardware resource
and performance constraints. Such ambitious nature of DeRy inevitably imposes
significant challenges, including, in the first place, the feasibility of its
solution. We strive to showcase that, through a dedicated paradigm proposed in
this paper, DeRy can be made not only possibly but practically efficiently.
Specifically, we conduct the partitions of all pre-trained networks jointly via
a cover set optimization, and derive a number of equivalence set, within each
of which the network blocks are treated as functionally equivalent and hence
interchangeable. The equivalence sets learned in this way, in turn, enable
picking and assembling blocks to customize networks subject to certain
constraints, which is achieved via solving an integer program backed up with a
training-free proxy to estimate the task performance. The reassembled models,
give rise to gratifying performances with the user-specified constraints
satisfied. We demonstrate that on ImageNet, the best reassemble model achieves
78.6% top-1 accuracy without fine-tuning, which could be further elevated to
83.2% with end-to-end training. Our code is available at
this https URL

该研究提出了一种深度模型重新组装（DeRy）技术，通过将预训练模型划分为等价集合并重新组合网络模块以满足硬件和性能限制，实现了通用模型的重复使用，并在 ImageNet 数据集上取得了令人满意的性能表现。