One of the primary challenges impeding the progress of Neural Architecture
Search (NAS) is its extensive reliance on exorbitant computational resources.
NAS benchmarks aim to simulate runs of NAS experiments at zero cost,
remediating the need for extensive compute. However, existing NAS benchmarks
use synthetic datasets and model proxies that make simplified assumptions about
the characteristics of these datasets and models, leading to unrealistic
evaluations. We present a technique that allows searching for training proxies
that reduce the cost of benchmark construction by significant margins, making
it possible to construct realistic NAS benchmarks for large-scale datasets.
Using this technique, we construct an open-source bi-objective NAS benchmark
for the ImageNet2012 dataset combined with the on-device performance of
accelerators, including GPUs, TPUs, and FPGAs. Through extensive
experimentation with various NAS optimizers and hardware platforms, we show
that the benchmark is accurate and allows searching for state-of-the-art
hardware-aware models at zero cost.

通过寻找训练代理，我们构建了一个开源的双目标神经架构搜索基准测试，用于大规模数据集，并结合了加速器的设备性能，包括 GPU、TPU 和 FPGA。通过与各种 NAS 优化器和硬件平台的广泛实验，我们展示了该基准测试的准确性，并且可以对硬件感知模型进行零成本的研究。