One-shot neural architecture search (NAS) has played a crucial role in making NAS methods computationally feasible in practice. Nevertheless, there is still a lack of understanding on how these weight-sharing algorithms exactly work due to the many factors controlling the dynamics of the process. In order to allow a scientific study of these components, we introduce a general framework for one-shot NAS that can be instantiated to many recently-introduced variants and introduce a general benchmarking framework that draws on the recent large-scale tabular benchmark NAS-Bench-101 for cheap anytime evaluations of one-shot NAS methods. To showcase the framework, we compare several state-of-the-art one-shot NAS methods, examine how sensitive they are to their hyperparameters and how they can be improved by tuning their hyperparameters, and compare their performance to that of blackbox optimizers for NAS-Bench-101.

本文提出了一个通用框架，用于一次性神经架构搜索，并引入了一个基于最近推出的大规模表格基准NAS-Bench-101的基准测试框架，以评估一次性NAS方法的廉价性能，并比较了几种最先进的一次性NAS方法，检查它们对超参数的敏感性，以及如何通过调整它们的超参数来改进它们的性能，并将它们的性能与NAS-Bench-101的黑盒优化器进行比较。

NAS-Bench-1Shot1: 一次性神经架构搜索的基准测试和解剖