Efficient evaluation of a network architecture drawn from a large search
space remains a key challenge in Neural Architecture Search (NAS). Vanilla NAS
evaluates each architecture by training from scratch, which gives the true
performance but is extremely time-consuming. Recently, one-shot NAS
substantially reduces the computation cost by training only one supernetwork,
a.k.a. supernet, to approximate the performance of every architecture in the
search space via weight-sharing. However, the performance estimation can be
very inaccurate due to the co-adaption among operations. In this paper, we
propose few-shot NAS that uses multiple supernetworks, called sub-supernet,
each covering different regions of the search space to alleviate the undesired
co-adaption. Compared to one-shot NAS, few-shot NAS improves the accuracy of
architecture evaluation with a small increase of evaluation cost. With only up
to 7 sub-supernets, few-shot NAS establishes new SoTAs: on ImageNet, it finds
models that reach 80.5% top-1 accuracy at 600 MB FLOPS and 77.5% top-1 accuracy
at 238 MFLOPS; on CIFAR10, it reaches 98.72% top-1 accuracy without using extra
data or transfer learning. In Auto-GAN, few-shot NAS outperforms the previously
published results by up to 20%. Extensive experiments show that few-shot NAS
significantly improves various one-shot methods, including 4 gradient-based and
6 search-based methods on 3 different tasks in NasBench-201 and
NasBench1-shot-1.

该研究提出了 few-shot NAS，利用多个超级网络（称为子超级网络）覆盖搜索空间的不同区域，以缓解操作之间的不良共适应，并证明其在多项任务中能够显著提高各种一次性方法的准确性，包括在 NasBench-201 和 NasBench1-shot-1 上的 3 个不同任务中的 4 个基于梯度的和 6 个基于搜索的方法，并在 ImageNet、CIFAR10 和 Auto-GAN 上取得最新水平。