Searching for the architecture cells is a dominant paradigm in NAS. However,
little attention has been devoted to the analysis of the cell-based search
spaces even though it is highly important for the continual development of NAS.
In this work, we conduct an empirical post-hoc analysis of architectures from
the popular cell-based search spaces and find that the existing search spaces
contain a high degree of redundancy: the architecture performance is minimally
sensitive to changes at large parts of the cells, and universally adopted
designs, like the explicit search for a reduction cell, significantly increase
the complexities but have very limited impact on the performance. Across
architectures found by a diverse set of search strategies, we consistently find
that the parts of the cells that do matter for architecture performance often
follow similar and simple patterns. By explicitly constraining cells to include
these patterns, randomly sampled architectures can match or even outperform the
state of the art. These findings cast doubts into our ability to discover truly
novel architectures in the existing cell-based search spaces, and inspire our
suggestions for improvement to guide future NAS research. Code is available at
this https URL.

本文在对流行的基于单元的搜索空间进行后期经验分析后发现，现有的搜索空间具有很高的冗余性，而单元中对性能影响最大的部分往往遵循类似的简单模式，并通过限制单元来包括这些模式，从而能找到一些能够匹配或胜过现有最先进模型的模型。