Slimmable networks are a family of neural networks that can instantly adjust
the runtime width. The width can be chosen from a predefined widths set to
adaptively optimize accuracy-efficiency trade-offs at runtime. In this work, we
propose a systematic approach to train universally slimmable networks
(US-Nets), extending slimmable networks to execute at arbitrary width, and
generalizing to networks both with and without batch normalization layers. We
further propose two improved training techniques for US-Nets, named the
sandwich rule and inplace distillation, to enhance training process and boost
testing accuracy. We show improved performance of universally slimmable
MobileNet v1 and MobileNet v2 on ImageNet classification task, compared with
individually trained ones and 4-switch slimmable network baselines. We also
evaluate the proposed US-Nets and improved training techniques on tasks of
image super-resolution and deep reinforcement learning. Extensive ablation
experiments on these representative tasks demonstrate the effectiveness of our
proposed methods. Our discovery opens up the possibility to directly evaluate
FLOPs-Accuracy spectrum of network architectures. Code and models are available
at: this https URL

该论文提出了一种名为 “通用可变宽度神经网络” 的系统方法，将可变宽度神经网络扩展到任意宽度，提出了两种改进的训练技术，能够提高训练过程和测试准确性；作者在 ImageNet 分类任务、图像超分辨率和深度强化学习任务上评估了其方法，证明其有效性，同时打开了直接评估网络结构 FLOPs-Accuracy 谱的可能性。