The growing interest in both the automation of machine learning and deep
learning has inevitably led to the development of a wide variety of automated
methods for neural architecture search. The choice of the network architecture
has proven to be critical, and many advances in deep learning spring from its
immediate improvements. However, deep learning techniques are computationally
intensive and their application requires a high level of domain knowledge.
Therefore, even partial automation of this process helps to make deep learning
more accessible to both researchers and practitioners. With this survey, we
provide a formalism which unifies and categorizes the landscape of existing
methods along with a detailed analysis that compares and contrasts the
different approaches. We achieve this via a comprehensive discussion of the
commonly adopted architecture search spaces and architecture optimization
algorithms based on principles of reinforcement learning and evolutionary
algorithms along with approaches that incorporate surrogate and one-shot
models. Additionally, we address the new research directions which include
constrained and multi-objective architecture search as well as automated data
augmentation, optimizer and activation function search.

本文综述了现有的方法，囊括了计算机领域中的神经体系结构搜索、自动化方法和深度学习等方面，内容包括常用的体系结构搜索空间和基于强化学习和进化算法的体系结构优化算法、模拟模型和单次模型。同时，文章对新的研究方向进行了探讨。

神经架构搜索综述

A Survey on Neural Architecture Search

Recent advances in neural architecture search (NAS) demand tremendous
computational resources, which makes it difficult to reproduce experiments and
imposes a barrier-to-entry to researchers without access to large-scale
computation. We aim to ameliorate these problems by introducing NAS-Bench-101,
the first public architecture dataset for NAS research. To build NAS-Bench-101,
we carefully constructed a compact, yet expressive, search space, exploiting
graph isomorphisms to identify 423k unique convolutional architectures. We
trained and evaluated all of these architectures multiple times on CIFAR-10 and
compiled the results into a large dataset of over 5 million trained models.
This allows researchers to evaluate the quality of a diverse range of models in
milliseconds by querying the pre-computed dataset. We demonstrate its utility
by analyzing the dataset as a whole and by benchmarking a range of architecture
optimization algorithms.

本文介绍了 NAS-Bench-101，这是第一个公开的神经架构搜索数据集，并描述了它是如何通过在 CIFAR-10 上训练和评估超过 423k 个卷积架构来构建的，旨在帮助研究者在毫秒内查询预计算的数据集来评估各种模型的质量。此外，作者还展示了该数据集作为整体的可用性以及对一系列架构优化算法进行基准测试的实用性。