Search space design is very critical to neural architecture search (NAS)
algorithms. We propose a fine-grained search space comprised of atomic blocks,
a minimal search unit that is much smaller than the ones used in recent NAS
algorithms. This search space allows a mix of operations by composing different
types of atomic blocks, while the search space in previous methods only allows
homogeneous operations. Based on this search space, we propose a resource-aware
architecture search framework which automatically assigns the computational
resources (e.g., output channel numbers) for each operation by jointly
considering the performance and the computational cost. In addition, to
accelerate the search process, we propose a dynamic network shrinkage technique
which prunes the atomic blocks with negligible influence on outputs on the fly.
Instead of a search-and-retrain two-stage paradigm, our method simultaneously
searches and trains the target architecture. Our method achieves
state-of-the-art performance under several FLOPs configurations on ImageNet
with a small searching cost. We open our entire codebase at:
this https URL

该文章提出了一种细粒度的搜索空间，有效地解决了神经体系结构搜索 (NAS) 中的搜索空间设计问题，其中包括资源感知的架构搜索框架和动态网络收缩技术，可同时进行搜索和训练，并在 ImageNet 上以小的搜索成本取得了最先进的性能。