If NAS methods are solutions, what is the problem? Most existing NAS methods
require two-stage parameter optimization. However, performance of the same
architecture in the two stages correlates poorly. In this work, we propose a
new problem definition for NAS, task-specific end-to-end, based on this
observation. We argue that given a computer vision task for which a NAS method
is expected, this definition can reduce the vaguely-defined NAS evaluation to
i) accuracy of this task and ii) the total computation consumed to finally
obtain a model with satisfying accuracy. Seeing that most existing methods do
not solve this problem directly, we propose DSNAS, an efficient differentiable
NAS framework that simultaneously optimizes architecture and parameters with a
low-biased Monte Carlo estimate. Child networks derived from DSNAS can be
deployed directly without parameter retraining. Comparing with two-stage
methods, DSNAS successfully discovers networks with comparable accuracy (74.4%)
on ImageNet in 420 GPU hours, reducing the total time by more than 34%. Our
implementation is available at this https URL

通过提出基于任务的端到端的 NAS 方法定义，我们提出了 DSNAS，一个高效的可微分 NAS 框架，不仅可以同时优化架构和参数，而且还可以减少模型训练时间，并在 ImageNet 数据集上达到了与现有方法同等的准确率。