Differentiable Neural Architecture Search (DARTS) is becoming more and more
popular among Neural Architecture Search (NAS) methods because of its high
search efficiency and low compute cost. However, the stability of DARTS is very
inferior, especially skip connections aggregation that leads to performance
collapse. Though existing methods leverage Hessian eigenvalues to alleviate
skip connections aggregation, they make DARTS unable to explore architectures
with better performance. In the paper, we propose operation-level progressive
differentiable neural architecture search (OPP-DARTS) to avoid skip connections
aggregation and explore better architectures simultaneously. We first divide
the search process into several stages during the search phase and increase
candidate operations into the search space progressively at the beginning of
each stage. It can effectively alleviate the unfair competition between
operations during the search phase of DARTS by offsetting the inherent unfair
advantage of the skip connection over other operations. Besides, to keep the
competition between operations relatively fair and select the operation from
the candidate operations set that makes training loss of the supernet largest.
The experiment results indicate that our method is effective and efficient. Our
method's performance on CIFAR-10 is superior to the architecture found by
standard DARTS, and the transferability of our method also surpasses standard
DARTS. We further demonstrate the robustness of our method on three simple
search spaces, i.e., S2, S3, S4, and the results show us that our method is
more robust than standard DARTS. Our code is available at
this https URL.

本文介绍了一种名为 OPP-DARTS 的操作级渐进可微分神经结构搜索算法，通过逐步增加候选操作来有效地避免跳跃连接聚合，并同时探索更好的架构。实验结果表明，我们的方法在 CIFAR-10 上的性能优于标准 DARTS，迁移性能也超越了标准 DARTS。此外，我们进一步展示了我们的方法在三个简单搜索空间上的鲁棒性优于标准 DARTS。

操作级渐进可微架构搜索

Operation-level Progressive Differentiable Architecture Search

This paper addresses the difficult problem of finding an optimal neural
architecture design for a given image classification task. We propose a method
that aggregates two main results of the previous state-of-the-art in neural
architecture search. These are, appealing to the strong sampling efficiency of
a search scheme based on sequential model-based optimization (SMBO), and
increasing training efficiency by sharing weights among sampled architectures.
Sequential search has previously demonstrated its capabilities to find
state-of-the-art neural architectures for image classification. However, its
computational cost remains high, even unreachable under modest computational
settings. Affording SMBO with weight-sharing alleviates this problem. On the
other hand, progressive search with SMBO is inherently greedy, as it leverages
a learned surrogate function to predict the validation error of neural
architectures. This prediction is directly used to rank the sampled neural
architectures. We propose to attenuate the greediness of the original SMBO
method by relaxing the role of the surrogate function so it predicts
architecture sampling probability instead. We demonstrate with experiments on
the CIFAR-10 dataset that our method, denominated Efficient progressive neural
architecture search (EPNAS), leads to increased search efficiency, while
retaining competitiveness of found architectures.

本文提出了一种称为 Efficient progressive neural architecture search (EPNAS) 的方法，将全新的序贯模型优化方法与权重共享结合，成功地提高了神经结构搜索的效率。