Neural architecture search (NAS) has gained immense popularity owing to its
ability to automate neural architecture design. A number of training-free
metrics are recently proposed to realize NAS without training, hence making NAS
more scalable. Despite their competitive empirical performances, a unified
theoretical understanding of these training-free metrics is lacking. As a
consequence, (a) the relationships among these metrics are unclear, (b) there
is no theoretical interpretation for their empirical performances, and (c)
there may exist untapped potential in existing training-free NAS, which
probably can be unveiled through a unified theoretical understanding. To this
end, this paper presents a unified theoretical analysis of gradient-based
training-free NAS, which allows us to (a) theoretically study their
relationships, (b) theoretically guarantee their generalization performances,
and (c) exploit our unified theoretical understanding to develop a novel
framework named hybrid NAS (HNAS) which consistently boosts training-free NAS
in a principled way. Remarkably, HNAS can enjoy the advantages of both
training-free (i.e., the superior search efficiency) and training-based (i.e.,
the remarkable search effectiveness) NAS, which we have demonstrated through
extensive experiments.

本文提出了一个统一的理论分析框架来研究基于梯度的免训练神经构架搜索方法，从而理论上研究它们之间的关系、保证它们的泛化性能并开发一种名为 Hybrid NAS (HNAS) 的新框架，它在原则上可以持续提高训练免费 NAS 的效果。