Neural architecture search (NAS) has dramatically advanced the development of
neural network design. We revisit the search space design in most previous NAS
methods and find the number and widths of blocks are set manually. However,
block counts and block widths determine the network scale (depth and width) and
make a great influence on both the accuracy and the model cost (FLOPs/latency).
In this paper, we propose to search block counts and block widths by designing
a densely connected search space, i.e., DenseNAS. The new search space is
represented as a dense super network, which is built upon our designed routing
blocks. In the super network, routing blocks are densely connected and we
search for the best path between them to derive the final architecture. We
further propose a chained cost estimation algorithm to approximate the model
cost during the search. Both the accuracy and model cost are optimized in
DenseNAS. For experiments on the MobileNetV2-based search space, DenseNAS
achieves 75.3% top-1 accuracy on ImageNet with only 361MB FLOPs and 17.9ms
latency on a single TITAN-XP. The larger model searched by DenseNAS achieves
76.1% accuracy with only 479M FLOPs. DenseNAS further promotes the ImageNet
classification accuracies of ResNet-18, -34 and -50-B by 1.5%, 0.5% and 0.3%
with 200M, 600M and 680M FLOPs reduction respectively. The related code is
available at this https URL

本文提出了一种名为 DenseNAS 的神经架构搜索方法，通过设计一种密集连接的搜索空间，在该空间中搜索最佳路径来获得最终的体系结构，并使用一种链式成本估算算法来近似模型成本，优化准确性和模型成本。实验结果表明，DenseNAS 在 ImageNet 上具有很高的分类准确率，并成功优化了 ResNet 等多个模型的分类性能。