This paper aims at enlarging the problem of Neural Architecture Search (NAS)
from Single-Path and Multi-Path Search to automated Mixed-Path Search. In
particular, we model the NAS problem as a sparse supernet using a new
continuous architecture representation with a mixture of sparsity constraints.
The sparse supernet enables us to automatically achieve sparsely-mixed paths
upon a compact set of nodes. To optimize the proposed sparse supernet, we
exploit a hierarchical accelerated proximal gradient algorithm within a
bi-level optimization framework. Extensive experiments on Convolutional Neural
Network and Recurrent Neural Network search demonstrate that the proposed
method is capable of searching for compact, general and powerful neural
architectures.

本论文旨在将神经架构搜索的问题从单路径和多路径搜索扩大到自动混合路径搜索。我们提出了一种新的连续架构表示方法，利用稀疏超网络和稀疏性约束来实现自动混合路径搜索。同时，我们采用了分层加速的近端梯度算法来优化所提出的稀疏超网络，并在卷积神经网络和循环神经网络搜索中进行了广泛的实验，证明了该方法能够搜索到紧凑、通用和强大的神经结构。