Recently, community has paid increasing attention on model scaling and
contributed to developing a model family with a wide spectrum of scales.
Current methods either simply resort to a one-shot NAS manner to construct a
non-structural and non-scalable model family or rely on a manual yet fixed
scaling strategy to scale an unnecessarily best base model. In this paper, we
bridge both two components and propose ScaleNet to jointly search base model
and scaling strategy so that the scaled large model can have more promising
performance. Concretely, we design a super-supernet to embody models with
different spectrum of sizes (e.g., FLOPs). Then, the scaling strategy can be
learned interactively with the base model via a Markov chain-based evolution
algorithm and generalized to develop even larger models. To obtain a decent
super-supernet, we design a hierarchical sampling strategy to enhance its
training sufficiency and alleviate the disturbance. Experimental results show
our scaled networks enjoy significant performance superiority on various FLOPs,
but with at least 2.53x reduction on search cost. Codes are available at
this https URL

本文提出了 ScaleNet 模型，结合了基模型和缩放策略的搜索，利用马尔科夫链进化算法互动学习基模型的缩放策略来发展具有更加优异性能的大型模型，实验结果表明我们的放大网络在各种 FLOPs 上具有显着的性能优势，但搜索成本至少降低了 2.53 倍。