DARTS search space (DSS) has become a canonical benchmark for NAS whereas
some emerging works pointed out the issue of narrow accuracy range and claimed
it would hurt the method ranking. We observe some recent studies already suffer
from this issue that overshadows the meaning of scores. In this work, we first
propose and orchestrate a suite of improvements to frame a larger and harder
DSS, termed LHD, while retaining high efficiency in search. We step forward to
renovate a LHD-based new benchmark, taking care of both discernibility and
accessibility. Specifically, we re-implement twelve baselines and evaluate them
across twelve conditions by combining two underexpolored influential factors:
transductive robustness and discretization policy, to reasonably construct a
benchmark upon multi-condition evaluation. Considering that the tabular
benchmarks are always insufficient to adequately evaluate the methods of neural
architecture search (NAS), our work can serve as a crucial basis for the future
progress of NAS. this https URL

本文提出了一种改进的、更大更难的基于 DARTS search space (LHD) 的基准测试，重新实现了 12 个基线并在 12 个条件下进行了评估，并探究了多种情况下的影响因素，可以作为神经结构搜索 (NAS) 未来进展的关键依据。