Conventional neural architecture search (NAS) approaches are based on
reinforcement learning or evolutionary strategy, which take more than 3000 GPU
hours to find a good model on CIFAR-10. We propose an efficient NAS approach
learning to search by gradient descent. Our approach represents the search
space as a directed acyclic graph (DAG). This DAG contains billions of
sub-graphs, each of which indicates a kind of neural architecture. To avoid
traversing all the possibilities of the sub-graphs, we develop a differentiable
sampler over the DAG. This sampler is learnable and optimized by the validation
loss after training the sampled architecture. In this way, our approach can be
trained in an end-to-end fashion by gradient descent, named Gradient-based
search using Differentiable Architecture Sampler (GDAS). In experiments, we can
finish one searching procedure in four GPU hours on CIFAR-10, and the
discovered model obtains a test error of 2.82\% with only 2.5M parameters,
which is on par with the state-of-the-art. Code is publicly available on
GitHub: this https URL

通过可微分采样器实现基于梯度下降的神经结构搜索方法，旨在在 CIFAR-10 数据集上提升搜索效率并获取与最新技术水平持平的测试准确率。