Differentiable architecture search (DAS) has made great progress in searching
for high-performance architectures with reduced computational cost. However,
DAS-based methods mainly focus on searching for a repeatable cell structure,
which is then stacked sequentially in multiple stages to form the networks.
This configuration significantly reduces the search space, and ignores the
importance of connections between the cells. To overcome this limitation, in
this paper, we propose a Hierarchical Differentiable Architecture Search
(H-DAS) that performs architecture search both at the cell level and at the
stage level. Specifically, the cell-level search space is relaxed so that the
networks can learn stage-specific cell structures. For the stage-level search,
we systematically study the architectures of stages, including the number of
cells in each stage and the connections between the cells. Based on insightful
observations, we design several search rules and losses, and mange to search
for better stage-level architectures. Such hierarchical search space greatly
improves the performance of the networks without introducing expensive search
cost. Extensive experiments on CIFAR10 and ImageNet demonstrate the
effectiveness of the proposed H-DAS. Moreover, the searched stage-level
architectures can be combined with the cell structures searched by existing DAS
methods to further boost the performance. Code is available at:
this https URL

该研究提出了层级可微架构搜索 (Hierarchical Differentiable Architecture Search, H-DAS) 方法，旨在在单元层面和阶段层面上进行架构搜索，通过放宽单元层面的搜索空间，设计阶段层面的搜索规则和损失，有效提升了网络性能，实验证明该方法的有效性。

Hierarchical Differentiable Architecture Search： 释放搜索空间的枷锁

Unchain the Search Space with Hierarchical Differentiable Architecture  Search

Neural Architecture Search (NAS) was first proposed to achieve
state-of-the-art performance through the discovery of new architecture
patterns, without human intervention. An over-reliance on expert knowledge in
the search space design has however led to increased performance (local optima)
without significant architectural breakthroughs, thus preventing truly novel
solutions from being reached. In this work we 1) are the first to investigate
casting NAS as a problem of finding the optimal network generator and 2) we
propose a new, hierarchical and graph-based search space capable of
representing an extremely large variety of network types, yet only requiring
few continuous hyper-parameters. This greatly reduces the dimensionality of the
problem, enabling the effective use of Bayesian Optimisation as a search
strategy. At the same time, we expand the range of valid architectures,
motivating a multi-objective learning approach. We demonstrate the
effectiveness of this strategy on six benchmark datasets and show that our
search space generates extremely lightweight yet highly competitive models.

本文提出了一种将神经结构搜索 (NAS) 看作寻找最优网络生成器的问题，同时提出了一种新的、层次化和基于图形的搜索空间，实现了在极少连续超参数的情况下表示极大多样性的网络类型。通过贝叶斯优化作为搜索策略，该方法有效地扩展了有效架构的范围，促进了多目标学习方法的发展， 并在六个基准数据集上进行了验证，在生成轻量级且高度竞争的模型方面表现出了有效性。