Neural architecture search (NAS) has gained immense popularity owing to its
ability to automate neural architecture design. A number of training-free
metrics are recently proposed to realize NAS without training, hence making NAS
more scalable. Despite their competitive empirical performances, a unified
theoretical understanding of these training-free metrics is lacking. As a
consequence, (a) the relationships among these metrics are unclear, (b) there
is no theoretical interpretation for their empirical performances, and (c)
there may exist untapped potential in existing training-free NAS, which
probably can be unveiled through a unified theoretical understanding. To this
end, this paper presents a unified theoretical analysis of gradient-based
training-free NAS, which allows us to (a) theoretically study their
relationships, (b) theoretically guarantee their generalization performances,
and (c) exploit our unified theoretical understanding to develop a novel
framework named hybrid NAS (HNAS) which consistently boosts training-free NAS
in a principled way. Remarkably, HNAS can enjoy the advantages of both
training-free (i.e., the superior search efficiency) and training-based (i.e.,
the remarkable search effectiveness) NAS, which we have demonstrated through
extensive experiments.

本文提出了一个统一的理论分析框架来研究基于梯度的免训练神经构架搜索方法，从而理论上研究它们之间的关系、保证它们的泛化性能并开发一种名为 Hybrid NAS (HNAS) 的新框架，它在原则上可以持续提高训练免费 NAS 的效果。

基于渐变的训练免费神经结构搜索的统一和增强

Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search

Differentiable Architecture Search (DARTS) has attracted extensive attention
due to its efficiency in searching for cell structures. DARTS mainly focuses on
the operation search and derives the cell topology from the operation weights.
However, the operation weights can not indicate the importance of cell topology
and result in poor topology rating correctness. To tackle this, we propose to
Decouple the Operation and Topology Search (DOTS), which decouples the topology
representation from operation weights and makes an explicit topology search.
DOTS is achieved by introducing a topology search space that contains
combinations of candidate edges. The proposed search space directly reflects
the search objective and can be easily extended to support a flexible number of
edges in the searched cell. Existing gradient-based NAS methods can be
incorporated into DOTS for further improvement by the topology search.
Considering that some operations (e.g., Skip-Connection) can affect the
topology, we propose a group operation search scheme to preserve
topology-related operations for a better topology search. The experiments on
CIFAR10/100 and ImageNet demonstrate that DOTS is an effective solution for
differentiable NAS.

本文提出了一种 Decouple the Operation and Topology Search（DOTS）的方法，该方法通过引入包含候选边的拓扑搜索空间实现了拓扑表示与操作权重的解耦，并且可以轻松扩展以支持搜索的单元中灵活数量的边。实验结果表明，DOTS 对可微架构搜索是一种有效的解决方案。