In one-shot weight sharing for NAS, the weights of each operation (at each
layer) are supposed to be identical for all architectures (paths) in the
supernet. However, this rules out the possibility of adjusting operation
weights to cater for different paths, which limits the reliability of the
evaluation results. In this paper, instead of counting on a single supernet, we
introduce $K$-shot supernets and take their weights for each operation as a
dictionary. The operation weight for each path is represented as a convex
combination of items in a dictionary with a simplex code. This enables a matrix
approximation of the stand-alone weight matrix with a higher rank ($K>1$). A
\textit{simplex-net} is introduced to produce architecture-customized code for
each path. As a result, all paths can adaptively learn how to share weights in
the $K$-shot supernets and acquire corresponding weights for better evaluation.
$K$-shot supernets and simplex-net can be iteratively trained, and we further
extend the search to the channel dimension. Extensive experiments on benchmark
datasets validate that K-shot NAS significantly improves the evaluation
accuracy of paths and thus brings in impressive performance improvements.

本文提出了一种基于 K-shot supernets 和 simplex-net 的 NAS 方法，可以自适应地学习权重共享，以获得更好的评估结果。实验结果表明，K-shot NAS 显著提高了路径的评估精度，并带来了令人印象深刻的性能提升。