We propose the width-resolution mutual learning method (MutualNet) to train a
network that is executable at dynamic resource constraints to achieve adaptive
accuracy-efficiency trade-offs at runtime. Our method trains a cohort of
sub-networks with different widths using different input resolutions to
mutually learn multi-scale representations for each sub-network. It achieves
consistently better ImageNet top-1 accuracy over the state-of-the-art adaptive
network US-Net under different computation constraints, and outperforms the
best compound scaled MobileNet in EfficientNet by 1.5%. The superiority of our
method is also validated on COCO object detection and instance segmentation as
well as transfer learning. Surprisingly, the training strategy of MutualNet can
also boost the performance of a single network, which substantially outperforms
the powerful AutoAugmentation in both efficiency (GPU search hours: 15000 vs.
0) and accuracy (ImageNet: 77.6% vs. 78.6%). Code is available at
https://github.com/taoyang1122/MutualNet.

提出了宽度 - 分辨率互学习方法 MutualNet，通过训练不同宽度的多个子网络来实现自适应准确性和效率之间的权衡，最终在不同计算约束下，通过互学习获得更好的 ImageNet top-1 精度。