It has become mainstream in computer vision and other machine learning
domains to reuse backbone networks pre-trained on large datasets as
preprocessors. Typically, the last layer is replaced by a shallow learning
machine of sorts; the newly-added classification head and (optionally) deeper
layers are fine-tuned on a new task. Due to its strong performance and
simplicity, a common pre-trained backbone network is ResNet152.However,
ResNet152 is relatively large and induces inference latency. In many cases, a
compact and efficient backbone with similar performance would be preferable
over a larger, slower one. This paper investigates techniques to reuse a
pre-trained backbone with the objective of creating a smaller and faster model.
Starting from a large ResNet152 backbone pre-trained on ImageNet, we first
reduce it from 51 blocks to 5 blocks, reducing its number of parameters and
FLOPs by more than 6 times, without significant performance degradation. Then,
we split the model after 3 blocks into several branches, while preserving the
same number of parameters and FLOPs, to create an ensemble of sub-networks to
improve performance. Our experiments on a large benchmark of $40$ image
classification datasets from various domains suggest that our techniques match
the performance (if not better) of ``classical backbone fine-tuning'' while
achieving a smaller model size and faster inference speed.

从一种大型的 ResNet152 骨干网络开始，本文通过将其从 51 个块降至 5 个块，将参数和 FLOPs 的数量降低了 6 倍以上，同时保留了性能，然后在 3 个块之后将模型分割为多个分支，创建一个子网络组合以提高性能。在 40 个图像分类数据集上的实验证明，本文的技术在与 “经典骨干微调” 相当甚至更好的性能的同时，实现了更小的模型尺寸和更快的推理速度。