The public model zoo containing enormous powerful pretrained model families
(e.g., ResNet/DeiT) has reached an unprecedented scope than ever, which
significantly contributes to the success of deep learning. As each model family
consists of pretrained models with diverse scales (e.g., DeiT-Ti/S/B), it
naturally arises a fundamental question of how to efficiently assemble these
readily available models in a family for dynamic accuracy-efficiency trade-offs
at runtime. To this end, we present Stitchable Neural Networks (SN-Net), a
novel scalable and efficient framework for model deployment which cheaply
produces numerous networks with different complexity and performance trade-offs
given a family of pretrained neural networks, which we call anchors.
Specifically, SN-Net splits the anchors across the blocks/layers and then
stitches them together with simple stitching layers to map the activations from
one anchor to another. With only a few epochs of training, SN-Net effectively
interpolates between the performance of anchors with varying scales. At
runtime, SN-Net can instantly adapt to dynamic resource constraints by
switching the stitching positions. Extensive experiments on ImageNet
classification demonstrate that SN-Net can obtain on-par or even better
performance than many individually trained networks while supporting diverse
deployment scenarios. For example, by stitching Swin Transformers, we challenge
hundreds of models in Timm model zoo with a single network. We believe this new
elastic model framework can serve as a strong baseline for further research in
wider communities.

提出了一种 Scalable 并有效的框架来对预训练神经网络进行动态准确率 - 效率权衡的组装，该方案称为 Stitchable Neural Networks（SN-Net）；该方案在 ImageNet 分类任务中取得了与已训练网络相当甚至更好的表现，并支持多种部署场景。